[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:08:14.212965  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:36783
I20260709 12:08:14.214881  9261 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:08:14.216181  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:14.236776  9267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:14.236763  9270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:14.237730  9268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:14.238437  9261 server_base.cc:1061] running on GCE node
I20260709 12:08:15.450627  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:15.450914  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:15.451179  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598895451154 us; error 0 us; skew 500 ppm
I20260709 12:08:15.452116  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:15.458464  9261 webserver.cc:533] Webserver started at http://127.9.11.126:34743/ using document root <none> and password file <none>
I20260709 12:08:15.459553  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:15.459781  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:15.460253  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:15.467200  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "75a097bc3547474393a99c0686c0ae5b"
format_stamp: "Formatted at 2026-07-09 12:08:15 on dist-test-slave-k289"
I20260709 12:08:15.476727  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:08:15.482889  9276 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:15.484288  9261 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:08:15.484681  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "75a097bc3547474393a99c0686c0ae5b"
format_stamp: "Formatted at 2026-07-09 12:08:15 on dist-test-slave-k289"
I20260709 12:08:15.485023  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:15.542919  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:15.544782  9261 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:08:15.545265  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:15.633100  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:36783
I20260709 12:08:15.633266  9327 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:36783 every 8 connection(s)
I20260709 12:08:15.643074  9328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:15.668474  9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b: Bootstrap starting.
I20260709 12:08:15.675951  9328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.678305  9328 log.cc:826] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b: Log is configured to *not* fsync() on all Append() calls
I20260709 12:08:15.684118  9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b: No bootstrap required, opened a new log
I20260709 12:08:15.707829  9328 raft_consensus.cc:359] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a097bc3547474393a99c0686c0ae5b" member_type: VOTER }
I20260709 12:08:15.708617  9328 raft_consensus.cc:385] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.708913  9328 raft_consensus.cc:740] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a097bc3547474393a99c0686c0ae5b, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.710115  9328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a097bc3547474393a99c0686c0ae5b" member_type: VOTER }
I20260709 12:08:15.710697  9328 raft_consensus.cc:399] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:08:15.711048  9328 raft_consensus.cc:493] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:08:15.711455  9328 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.716398  9328 raft_consensus.cc:515] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a097bc3547474393a99c0686c0ae5b" member_type: VOTER }
I20260709 12:08:15.717317  9328 leader_election.cc:304] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75a097bc3547474393a99c0686c0ae5b; no voters: 
I20260709 12:08:15.719559  9328 leader_election.cc:290] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:08:15.719959  9331 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:15.723971  9331 raft_consensus.cc:697] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 1 LEADER]: Becoming Leader. State: Replica: 75a097bc3547474393a99c0686c0ae5b, State: Running, Role: LEADER
I20260709 12:08:15.725184  9331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a097bc3547474393a99c0686c0ae5b" member_type: VOTER }
I20260709 12:08:15.726468  9328 sys_catalog.cc:565] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:08:15.739288  9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75a097bc3547474393a99c0686c0ae5b. Latest consensus state: current_term: 1 leader_uuid: "75a097bc3547474393a99c0686c0ae5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a097bc3547474393a99c0686c0ae5b" member_type: VOTER } }
I20260709 12:08:15.739848  9333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75a097bc3547474393a99c0686c0ae5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a097bc3547474393a99c0686c0ae5b" member_type: VOTER } }
I20260709 12:08:15.740289  9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:15.740413  9333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:15.744853  9340 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:08:15.757735  9340 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:08:15.761373  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:08:15.778699  9340 catalog_manager.cc:1383] Generated new cluster ID: 7413701f9cf540049b7653bc68b72923
I20260709 12:08:15.779112  9340 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:08:15.797498  9340 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:08:15.800124  9340 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:08:15.820827  9340 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b: Generated new TSK 0
I20260709 12:08:15.822310  9340 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:08:15.893287  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:15.903285  9350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:15.905261  9351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:15.909276  9261 server_base.cc:1061] running on GCE node
W20260709 12:08:15.909811  9353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:15.910902  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:15.911206  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:15.911392  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598895911366 us; error 0 us; skew 500 ppm
I20260709 12:08:15.912041  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:15.917737  9261 webserver.cc:533] Webserver started at http://127.9.11.65:46715/ using document root <none> and password file <none>
I20260709 12:08:15.918432  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:15.918694  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:15.919101  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:15.920511  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "9881f5442a5f43c689c38301833d86c4"
format_stamp: "Formatted at 2026-07-09 12:08:15 on dist-test-slave-k289"
I20260709 12:08:15.928399  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:08:15.934351  9358 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:15.935527  9261 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:08:15.935889  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "9881f5442a5f43c689c38301833d86c4"
format_stamp: "Formatted at 2026-07-09 12:08:15 on dist-test-slave-k289"
I20260709 12:08:15.936250  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:15.956451  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:15.958140  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:15.962951  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:15.969756  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:15.970041  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:15.970404  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:15.970582  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.101496  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:35381
I20260709 12:08:16.101575  9420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:35381 every 8 connection(s)
I20260709 12:08:16.109939  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:16.124500  9425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:16.123730  9426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:16.135171  9261 server_base.cc:1061] running on GCE node
W20260709 12:08:16.135646  9429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:16.136971  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:16.137271  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:16.137493  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598896137477 us; error 0 us; skew 500 ppm
I20260709 12:08:16.138376  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:16.143110  9261 webserver.cc:533] Webserver started at http://127.9.11.66:39559/ using document root <none> and password file <none>
I20260709 12:08:16.143848  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:16.143973  9421 heartbeater.cc:344] Connected to a master server at 127.9.11.126:36783
I20260709 12:08:16.144155  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:16.144559  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:16.144564  9421 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:16.145973  9421 heartbeater.cc:507] Master 127.9.11.126:36783 requested a full tablet report, sending...
I20260709 12:08:16.146340  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "7f13e6361ca548a08c36d273c6084d67"
format_stamp: "Formatted at 2026-07-09 12:08:16 on dist-test-slave-k289"
I20260709 12:08:16.149199  9293 ts_manager.cc:194] Registered new tserver with Master: 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381)
I20260709 12:08:16.151659  9293 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40368
I20260709 12:08:16.154111  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.001s	sys 0.005s
I20260709 12:08:16.159739  9433 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.160853  9261 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:08:16.161208  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "7f13e6361ca548a08c36d273c6084d67"
format_stamp: "Formatted at 2026-07-09 12:08:16 on dist-test-slave-k289"
I20260709 12:08:16.161586  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:16.205155  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:16.207229  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:16.209067  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:16.212466  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:16.212657  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.213020  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:16.213176  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.329039  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:32955
I20260709 12:08:16.329164  9495 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:32955 every 8 connection(s)
I20260709 12:08:16.335888  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:16.344543  9499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:16.348084  9500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:16.352859  9496 heartbeater.cc:344] Connected to a master server at 127.9.11.126:36783
W20260709 12:08:16.353026  9502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:16.353336  9496 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:16.354415  9496 heartbeater.cc:507] Master 127.9.11.126:36783 requested a full tablet report, sending...
I20260709 12:08:16.355247  9261 server_base.cc:1061] running on GCE node
I20260709 12:08:16.356420  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:16.356683  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:16.356876  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598896356856 us; error 0 us; skew 500 ppm
I20260709 12:08:16.357081  9293 ts_manager.cc:194] Registered new tserver with Master: 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:16.357648  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:16.359145  9293 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40372
I20260709 12:08:16.363683  9261 webserver.cc:533] Webserver started at http://127.9.11.67:39153/ using document root <none> and password file <none>
I20260709 12:08:16.364319  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:16.364617  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:16.364928  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:16.366127  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "e494877066cb40059085eceafc819c88"
format_stamp: "Formatted at 2026-07-09 12:08:16 on dist-test-slave-k289"
I20260709 12:08:16.371346  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:08:16.375346  9507 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.376329  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:08:16.376636  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "e494877066cb40059085eceafc819c88"
format_stamp: "Formatted at 2026-07-09 12:08:16 on dist-test-slave-k289"
I20260709 12:08:16.376929  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:16.390972  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:16.392464  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:16.395395  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:16.399366  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:16.399552  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.399865  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:16.400029  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:16.522336  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:39087
I20260709 12:08:16.523033  9569 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:39087 every 8 connection(s)
I20260709 12:08:16.543198  9570 heartbeater.cc:344] Connected to a master server at 127.9.11.126:36783
I20260709 12:08:16.543653  9570 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:16.544652  9570 heartbeater.cc:507] Master 127.9.11.126:36783 requested a full tablet report, sending...
I20260709 12:08:16.547034  9293 ts_manager.cc:194] Registered new tserver with Master: e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:16.547513  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018700685s
I20260709 12:08:16.548923  9293 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40384
I20260709 12:08:16.617561  9293 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40392:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:08:16.623948  9293 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 12:08:16.747445  9341 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:16.751277  9535 tablet_service.cc:1511] Processing CreateTablet for tablet 8bea185737654ec19e81900b0fb53191 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:08:16.756364  9535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bea185737654ec19e81900b0fb53191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.757591  9534 tablet_service.cc:1511] Processing CreateTablet for tablet 9dcb500384384968ae7f939bef8cdbfc (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:08:16.759478  9533 tablet_service.cc:1511] Processing CreateTablet for tablet a5da7e262d3343c7917047b2c07bb630 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:08:16.760739  9532 tablet_service.cc:1511] Processing CreateTablet for tablet 12b6f457c9cc43139072ace0befcd7c6 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:08:16.761631  9530 tablet_service.cc:1511] Processing CreateTablet for tablet 4dc54a439bef47fe9318f4f1fa6edcfd (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:08:16.763195  9531 tablet_service.cc:1511] Processing CreateTablet for tablet 157120855af2424da85e64e0c5d847a0 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:08:16.769126  9533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5da7e262d3343c7917047b2c07bb630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.773422  9534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dcb500384384968ae7f939bef8cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.774269  9532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b6f457c9cc43139072ace0befcd7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.775070  9530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dc54a439bef47fe9318f4f1fa6edcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.768173  9529 tablet_service.cc:1511] Processing CreateTablet for tablet 41bad849598b42a283ba311a077d4c41 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:08:16.781409  9531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157120855af2424da85e64e0c5d847a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.781170  9527 tablet_service.cc:1511] Processing CreateTablet for tablet 56139526e7f64ab582bf591daa967028 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:08:16.790081  9529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bad849598b42a283ba311a077d4c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.787082  9528 tablet_service.cc:1511] Processing CreateTablet for tablet f689a6b646c5419cb6432592e047a1b8 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:08:16.797297  9454 tablet_service.cc:1511] Processing CreateTablet for tablet f689a6b646c5419cb6432592e047a1b8 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:08:16.790671  9461 tablet_service.cc:1511] Processing CreateTablet for tablet 8bea185737654ec19e81900b0fb53191 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:08:16.803485  9527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56139526e7f64ab582bf591daa967028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.803804  9456 tablet_service.cc:1511] Processing CreateTablet for tablet 4dc54a439bef47fe9318f4f1fa6edcfd (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:08:16.804891  9454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f689a6b646c5419cb6432592e047a1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.807767  9453 tablet_service.cc:1511] Processing CreateTablet for tablet 56139526e7f64ab582bf591daa967028 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:08:16.798431  9460 tablet_service.cc:1511] Processing CreateTablet for tablet 9dcb500384384968ae7f939bef8cdbfc (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:08:16.802253  9457 tablet_service.cc:1511] Processing CreateTablet for tablet 157120855af2424da85e64e0c5d847a0 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:08:16.816174  9461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bea185737654ec19e81900b0fb53191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.808499  9455 tablet_service.cc:1511] Processing CreateTablet for tablet 41bad849598b42a283ba311a077d4c41 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:08:16.821388  9386 tablet_service.cc:1511] Processing CreateTablet for tablet 8bea185737654ec19e81900b0fb53191 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:08:16.804245  9459 tablet_service.cc:1511] Processing CreateTablet for tablet a5da7e262d3343c7917047b2c07bb630 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:08:16.809192  9458 tablet_service.cc:1511] Processing CreateTablet for tablet 12b6f457c9cc43139072ace0befcd7c6 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:08:16.825645  9528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f689a6b646c5419cb6432592e047a1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.829739  9386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bea185737654ec19e81900b0fb53191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.836014  9385 tablet_service.cc:1511] Processing CreateTablet for tablet 9dcb500384384968ae7f939bef8cdbfc (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:08:16.834847  9384 tablet_service.cc:1511] Processing CreateTablet for tablet a5da7e262d3343c7917047b2c07bb630 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:08:16.840600  9382 tablet_service.cc:1511] Processing CreateTablet for tablet 157120855af2424da85e64e0c5d847a0 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:08:16.840804  9383 tablet_service.cc:1511] Processing CreateTablet for tablet 12b6f457c9cc43139072ace0befcd7c6 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:08:16.845202  9385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dcb500384384968ae7f939bef8cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.853293  9381 tablet_service.cc:1511] Processing CreateTablet for tablet 4dc54a439bef47fe9318f4f1fa6edcfd (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:08:16.853351  9380 tablet_service.cc:1511] Processing CreateTablet for tablet 41bad849598b42a283ba311a077d4c41 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:08:16.859189  9384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5da7e262d3343c7917047b2c07bb630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.861029  9380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bad849598b42a283ba311a077d4c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.859181  9379 tablet_service.cc:1511] Processing CreateTablet for tablet f689a6b646c5419cb6432592e047a1b8 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:08:16.864163  9382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157120855af2424da85e64e0c5d847a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.866163  9378 tablet_service.cc:1511] Processing CreateTablet for tablet 56139526e7f64ab582bf591daa967028 (DEFAULT_TABLE table=test-workload [id=d448fda5caac44f28d1224b8076cd463]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:08:16.876474  9460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dcb500384384968ae7f939bef8cdbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.879558  9455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bad849598b42a283ba311a077d4c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.884507  9457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157120855af2424da85e64e0c5d847a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.886974  9458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b6f457c9cc43139072ace0befcd7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.883492  9383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b6f457c9cc43139072ace0befcd7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.890276  9459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5da7e262d3343c7917047b2c07bb630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.911886  9453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56139526e7f64ab582bf591daa967028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.901225  9381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dc54a439bef47fe9318f4f1fa6edcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.921144  9379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f689a6b646c5419cb6432592e047a1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.929224  9456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dc54a439bef47fe9318f4f1fa6edcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:16.938845  9378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56139526e7f64ab582bf591daa967028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:17.011118  9588 tablet_bootstrap.cc:492] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.028534  9588 tablet_bootstrap.cc:654] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.054253  9589 tablet_bootstrap.cc:492] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.067814  9590 tablet_bootstrap.cc:492] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.099608  9590 tablet_bootstrap.cc:654] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.115815  9589 tablet_bootstrap.cc:654] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.125030  9588 tablet_bootstrap.cc:492] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.128075  9588 ts_tablet_manager.cc:1403] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.121s	user 0.011s	sys 0.020s
I20260709 12:08:17.134037  9590 tablet_bootstrap.cc:492] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.134560  9590 ts_tablet_manager.cc:1403] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.068s	user 0.004s	sys 0.012s
I20260709 12:08:17.135092  9588 raft_consensus.cc:359] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.136045  9588 raft_consensus.cc:385] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.136411  9588 raft_consensus.cc:740] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.137125  9589 tablet_bootstrap.cc:492] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.137728  9589 ts_tablet_manager.cc:1403] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.010s
I20260709 12:08:17.137578  9588 consensus_queue.cc:260] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.137820  9590 raft_consensus.cc:359] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.138682  9590 raft_consensus.cc:385] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.139019  9590 raft_consensus.cc:740] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.139887  9590 consensus_queue.cc:260] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.142409  9589 raft_consensus.cc:359] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.143647  9589 raft_consensus.cc:385] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.144196  9589 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.145797  9589 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.147217  9570 heartbeater.cc:499] Master 127.9.11.126:36783 was elected leader, sending a full tablet report...
I20260709 12:08:17.148542  9588 ts_tablet_manager.cc:1434] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 12:08:17.150053  9588 tablet_bootstrap.cc:492] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.155822  9421 heartbeater.cc:499] Master 127.9.11.126:36783 was elected leader, sending a full tablet report...
I20260709 12:08:17.158824  9588 tablet_bootstrap.cc:654] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.168592  9590 ts_tablet_manager.cc:1434] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.034s	user 0.012s	sys 0.011s
I20260709 12:08:17.169900  9590 tablet_bootstrap.cc:492] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.176074  9589 ts_tablet_manager.cc:1434] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.009s
I20260709 12:08:17.176846  9590 tablet_bootstrap.cc:654] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.177138  9496 heartbeater.cc:499] Master 127.9.11.126:36783 was elected leader, sending a full tablet report...
I20260709 12:08:17.186892  9589 tablet_bootstrap.cc:492] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.194895  9590 tablet_bootstrap.cc:492] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.195667  9590 ts_tablet_manager.cc:1403] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.000s
I20260709 12:08:17.199672  9590 raft_consensus.cc:359] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.200192  9588 tablet_bootstrap.cc:492] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.200500  9590 raft_consensus.cc:385] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.200892  9590 raft_consensus.cc:740] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.201129  9589 tablet_bootstrap.cc:654] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.201030  9588 ts_tablet_manager.cc:1403] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.051s	user 0.007s	sys 0.012s
I20260709 12:08:17.201738  9590 consensus_queue.cc:260] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.204208  9590 ts_tablet_manager.cc:1434] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:08:17.206961  9590 tablet_bootstrap.cc:492] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.209501  9588 raft_consensus.cc:359] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.211624  9588 raft_consensus.cc:385] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.212018  9588 raft_consensus.cc:740] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.213231  9588 consensus_queue.cc:260] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.217079  9588 ts_tablet_manager.cc:1434] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.004s
I20260709 12:08:17.218150  9588 tablet_bootstrap.cc:492] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.223944  9590 tablet_bootstrap.cc:654] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.230532  9589 tablet_bootstrap.cc:492] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.235343  9589 ts_tablet_manager.cc:1403] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.010s
I20260709 12:08:17.238281  9590 tablet_bootstrap.cc:492] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.238965  9590 ts_tablet_manager.cc:1403] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.006s
I20260709 12:08:17.243996  9590 raft_consensus.cc:359] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.244844  9590 raft_consensus.cc:385] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.245206  9590 raft_consensus.cc:740] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.245009  9589 raft_consensus.cc:359] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.246078  9589 raft_consensus.cc:385] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.246587  9589 raft_consensus.cc:740] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.246147  9590 consensus_queue.cc:260] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.247725  9589 consensus_queue.cc:260] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.249504  9590 ts_tablet_manager.cc:1434] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:08:17.250571  9590 tablet_bootstrap.cc:492] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.250654  9589 ts_tablet_manager.cc:1434] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:08:17.256158  9589 tablet_bootstrap.cc:492] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.258360  9590 tablet_bootstrap.cc:654] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.258512  9588 tablet_bootstrap.cc:654] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.270560  9589 tablet_bootstrap.cc:654] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.272696  9588 tablet_bootstrap.cc:492] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.272696  9590 tablet_bootstrap.cc:492] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.273396  9588 ts_tablet_manager.cc:1403] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.056s	user 0.019s	sys 0.002s
I20260709 12:08:17.273407  9590 ts_tablet_manager.cc:1403] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.012s
I20260709 12:08:17.274914  9596 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.275573  9596 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.277503  9588 raft_consensus.cc:359] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.277396  9590 raft_consensus.cc:359] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.278261  9588 raft_consensus.cc:385] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.278517  9590 raft_consensus.cc:385] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.278735  9588 raft_consensus.cc:740] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.279631  9596 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
W20260709 12:08:17.280143  9571 tablet.cc:2406] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:08:17.285334  9590 raft_consensus.cc:740] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.286119  9588 consensus_queue.cc:260] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.287746  9590 consensus_queue.cc:260] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.292698  9590 ts_tablet_manager.cc:1434] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:08:17.289583  9588 ts_tablet_manager.cc:1434] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.002s
I20260709 12:08:17.296314  9590 tablet_bootstrap.cc:492] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.296607  9588 tablet_bootstrap.cc:492] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.305560  9588 tablet_bootstrap.cc:654] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.312323  9589 tablet_bootstrap.cc:492] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.312947  9589 ts_tablet_manager.cc:1403] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.006s
I20260709 12:08:17.316834  9589 raft_consensus.cc:359] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.317734  9589 raft_consensus.cc:385] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.318041  9589 raft_consensus.cc:740] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.318217  9588 tablet_bootstrap.cc:492] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.318795  9588 ts_tablet_manager.cc:1403] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.009s
I20260709 12:08:17.318783  9589 consensus_queue.cc:260] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.321846  9589 ts_tablet_manager.cc:1434] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:08:17.322960  9589 tablet_bootstrap.cc:492] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.322546  9588 raft_consensus.cc:359] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.323621  9588 raft_consensus.cc:385] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.324112  9588 raft_consensus.cc:740] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.326184  9588 consensus_queue.cc:260] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.330116  9590 tablet_bootstrap.cc:654] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.330116  9588 ts_tablet_manager.cc:1434] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 12:08:17.336258  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:17.339304  9588 tablet_bootstrap.cc:492] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88: Bootstrap starting.
W20260709 12:08:17.340445  9434 leader_election.cc:343] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.349869  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.351202  9588 tablet_bootstrap.cc:654] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:17.351960  9434 leader_election.cc:343] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e494877066cb40059085eceafc819c88 (127.9.11.67:39087): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.352415  9589 tablet_bootstrap.cc:654] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.352574  9434 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67; no voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88
I20260709 12:08:17.353741  9596 raft_consensus.cc:2749] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:17.363334  9595 raft_consensus.cc:493] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.363888  9588 tablet_bootstrap.cc:492] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.363935  9595 raft_consensus.cc:515] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.364831  9588 ts_tablet_manager.cc:1403] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.006s
I20260709 12:08:17.370019  9595 leader_election.cc:290] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.375450  9588 raft_consensus.cc:359] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.376691  9588 raft_consensus.cc:385] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.377149  9588 raft_consensus.cc:740] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.384543  9590 tablet_bootstrap.cc:492] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.385041  9590 ts_tablet_manager.cc:1403] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.013s
I20260709 12:08:17.388005  9590 raft_consensus.cc:359] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.388978  9590 raft_consensus.cc:385] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.389349  9590 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.390204  9590 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.378306  9588 consensus_queue.cc:260] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.399128  9595 raft_consensus.cc:493] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.400475  9595 raft_consensus.cc:515] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.401599  9594 raft_consensus.cc:493] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.402228  9594 raft_consensus.cc:515] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.403632  9588 ts_tablet_manager.cc:1434] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.004s
I20260709 12:08:17.403832  9595 leader_election.cc:290] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.405597  9594 leader_election.cc:290] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.408520  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.409353  9545 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 0.
I20260709 12:08:17.410758  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.412029  9544 raft_consensus.cc:2468] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 0.
I20260709 12:08:17.413925  9589 tablet_bootstrap.cc:492] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.414208  9588 tablet_bootstrap.cc:492] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.414628  9589 ts_tablet_manager.cc:1403] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.031s
I20260709 12:08:17.393440  9590 ts_tablet_manager.cc:1434] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:08:17.416572  9590 tablet_bootstrap.cc:492] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.418658  9589 raft_consensus.cc:359] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.419471  9589 raft_consensus.cc:385] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.419823  9589 raft_consensus.cc:740] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.423822  9589 consensus_queue.cc:260] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.425608  9471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.428335  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.427227  9471 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 0.
I20260709 12:08:17.433614  9588 tablet_bootstrap.cc:654] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.443573  9589 ts_tablet_manager.cc:1434] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:08:17.443300  9359 leader_election.cc:304] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
W20260709 12:08:17.445348  9362 leader_election.cc:343] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.447046  9595 raft_consensus.cc:2804] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.450470  9589 tablet_bootstrap.cc:492] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.447677  9595 raft_consensus.cc:493] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.453367  9594 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.454241  9590 tablet_bootstrap.cc:654] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.454020  9594 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.453351  9595 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.458175  9594 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.446833  9362 leader_election.cc:304] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:17.470628  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:17.471516  9396 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 0.
I20260709 12:08:17.474051  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:17.473361  9594 raft_consensus.cc:493] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.476248  9508 leader_election.cc:304] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:17.476857  9594 raft_consensus.cc:515] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.483086  9594 leader_election.cc:290] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.483683  9471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.485742  9471 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 0.
I20260709 12:08:17.488018  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.486573  9594 raft_consensus.cc:2804] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:08:17.482607  9362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:32955, user_credentials={real_user=slave}} blocked reactor thread for 45424us
I20260709 12:08:17.490854  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dc54a439bef47fe9318f4f1fa6edcfd" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.492247  9594 raft_consensus.cc:493] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:08:17.496555  9511 leader_election.cc:343] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.494479  9594 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.497218  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dc54a439bef47fe9318f4f1fa6edcfd" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
W20260709 12:08:17.497591  9511 leader_election.cc:343] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.498277  9617 raft_consensus.cc:2804] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.498970  9617 raft_consensus.cc:493] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:08:17.499382  9508 leader_election.cc:343] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.492007  9595 raft_consensus.cc:515] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.500274  9508 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e494877066cb40059085eceafc819c88; no voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:17.501869  9588 tablet_bootstrap.cc:492] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.502769  9590 tablet_bootstrap.cc:492] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.499756  9617 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.503433  9595 leader_election.cc:290] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.504508  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:17.507810  9469 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.507671  9594 raft_consensus.cc:515] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.511543  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:17.512244  9470 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.505246  9590 ts_tablet_manager.cc:1403] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.089s	user 0.016s	sys 0.010s
I20260709 12:08:17.515302  9469 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:17.505656  9588 ts_tablet_manager.cc:1403] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.092s	user 0.010s	sys 0.016s
I20260709 12:08:17.515774  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:17.516366  9544 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.516789  9362 leader_election.cc:304] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:17.518453  9589 tablet_bootstrap.cc:654] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.506377  9620 raft_consensus.cc:2749] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:17.520768  9470 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 1.
I20260709 12:08:17.522781  9511 leader_election.cc:304] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:17.522643  9588 raft_consensus.cc:359] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.523640  9588 raft_consensus.cc:385] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.524298  9588 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.525346  9588 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.526085  9544 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:17.527124  9620 raft_consensus.cc:2804] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:17.529448  9620 raft_consensus.cc:697] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:17.530618  9620 consensus_queue.cc:237] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.532786  9589 tablet_bootstrap.cc:492] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.527724  9595 raft_consensus.cc:2804] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:17.521382  9594 leader_election.cc:290] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.533346  9589 ts_tablet_manager.cc:1403] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.012s
I20260709 12:08:17.527879  9590 raft_consensus.cc:359] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
W20260709 12:08:17.528496  9508 leader_election.cc:343] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.534977  9590 raft_consensus.cc:385] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.536154  9590 raft_consensus.cc:740] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.535487  9508 leader_election.cc:304] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e494877066cb40059085eceafc819c88; no voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:17.537530  9594 raft_consensus.cc:2749] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:17.541569  9589 raft_consensus.cc:359] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.545153  9589 raft_consensus.cc:385] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.545598  9589 raft_consensus.cc:740] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.546307  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:17.547178  9395 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.547053  9589 consensus_queue.cc:260] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.564183  9395 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 1.
I20260709 12:08:17.553241  9590 consensus_queue.cc:260] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.515339  9617 raft_consensus.cc:515] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.570842  9595 raft_consensus.cc:697] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:17.573626  9589 ts_tablet_manager.cc:1434] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.007s
I20260709 12:08:17.576757  9589 tablet_bootstrap.cc:492] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.579658  9595 consensus_queue.cc:237] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:17.585736  9588 ts_tablet_manager.cc:1434] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.070s	user 0.018s	sys 0.025s
I20260709 12:08:17.587723  9588 tablet_bootstrap.cc:492] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.593103  9623 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.594148  9623 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.598047  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:17.600139  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
W20260709 12:08:17.600351  9362 leader_election.cc:343] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.600854  9395 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 0.
I20260709 12:08:17.600829  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.601214  9624 raft_consensus.cc:493] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.601927  9470 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 0.
I20260709 12:08:17.602236  9508 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:17.603022  9623 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.603979  9622 raft_consensus.cc:2804] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.606663  9590 ts_tablet_manager.cc:1434] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.091s	user 0.021s	sys 0.018s
I20260709 12:08:17.608183  9622 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.608477  9590 tablet_bootstrap.cc:492] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.608776  9622 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.601940  9624 raft_consensus.cc:515] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.614010  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:17.615901  9544 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.619690  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.624234  9470 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 0.
I20260709 12:08:17.611429  9617 leader_election.cc:290] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.628778  9362 leader_election.cc:304] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:17.631598  9589 tablet_bootstrap.cc:654] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.632429  9624 leader_election.cc:290] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.633448  9617 raft_consensus.cc:2804] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.635322  9617 raft_consensus.cc:493] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.635735  9617 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.634475  9544 raft_consensus.cc:2468] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:17.634382  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.640050  9359 leader_election.cc:304] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:17.640998  9624 raft_consensus.cc:2804] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:08:17.641773  9359 leader_election.cc:343] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e494877066cb40059085eceafc819c88 (127.9.11.67:39087): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.641937  9624 raft_consensus.cc:697] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:17.643599  9617 raft_consensus.cc:515] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.644462  9588 tablet_bootstrap.cc:654] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.643745  9624 consensus_queue.cc:237] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.646858  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:17.647714  9470 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.649412  9622 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.650957  9590 tablet_bootstrap.cc:654] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.658349  9588 tablet_bootstrap.cc:492] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.658533  9470 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:17.658907  9588 ts_tablet_manager.cc:1403] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.000s
I20260709 12:08:17.660619  9622 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.661489  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:17.661978  9362 leader_election.cc:304] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:17.662149  9395 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.664875  9625 raft_consensus.cc:2804] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:17.665437  9625 raft_consensus.cc:697] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:17.666293  9625 consensus_queue.cc:237] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.670529  9589 tablet_bootstrap.cc:492] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.671105  9589 ts_tablet_manager.cc:1403] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.001s
I20260709 12:08:17.671114  9588 raft_consensus.cc:359] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.674019  9588 raft_consensus.cc:385] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.674386  9588 raft_consensus.cc:740] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.673516  9395 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 1.
I20260709 12:08:17.675508  9588 consensus_queue.cc:260] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.667649  9290 catalog_manager.cc:5697] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 0 to 1, leader changed from <none> to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:17.677330  9508 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:17.678395  9620 raft_consensus.cc:2804] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:17.677047  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:17.680217  9620 raft_consensus.cc:697] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:17.683643  9620 consensus_queue.cc:237] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.684916  9588 ts_tablet_manager.cc:1434] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:08:17.686097  9588 tablet_bootstrap.cc:492] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.686563  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:17.687198  9544 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.686335  9293 catalog_manager.cc:5697] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:17.692972  9470 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.689996  9617 leader_election.cc:290] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.699077  9589 raft_consensus.cc:359] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.700038  9589 raft_consensus.cc:385] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.700416  9589 raft_consensus.cc:740] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.701375  9589 consensus_queue.cc:260] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.716524  9590 tablet_bootstrap.cc:492] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.717154  9590 ts_tablet_manager.cc:1403] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.109s	user 0.008s	sys 0.018s
I20260709 12:08:17.717586  9544 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:17.723389  9589 ts_tablet_manager.cc:1434] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 12:08:17.725530  9589 tablet_bootstrap.cc:492] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.732317  9590 raft_consensus.cc:359] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.734197  9590 raft_consensus.cc:385] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.736016  9590 raft_consensus.cc:740] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.738303  9588 tablet_bootstrap.cc:654] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.741567  9589 tablet_bootstrap.cc:654] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.743075  9590 consensus_queue.cc:260] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.746908  9470 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 1.
I20260709 12:08:17.752094  9590 ts_tablet_manager.cc:1434] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 12:08:17.754106  9588 tablet_bootstrap.cc:492] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.754710  9588 ts_tablet_manager.cc:1403] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.069s	user 0.011s	sys 0.002s
I20260709 12:08:17.757781  9588 raft_consensus.cc:359] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.758504  9588 raft_consensus.cc:385] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.759249  9590 tablet_bootstrap.cc:492] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
W20260709 12:08:17.760317  9341 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:17.758800  9588 raft_consensus.cc:740] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.767804  9588 consensus_queue.cc:260] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.771626  9588 ts_tablet_manager.cc:1434] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 12:08:17.772923  9588 tablet_bootstrap.cc:492] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88: Bootstrap starting.
I20260709 12:08:17.781126  9590 tablet_bootstrap.cc:654] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.781235  9589 tablet_bootstrap.cc:492] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.781919  9589 ts_tablet_manager.cc:1403] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.057s	user 0.026s	sys 0.017s
I20260709 12:08:17.790376  9589 raft_consensus.cc:359] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.791560  9624 raft_consensus.cc:493] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.792119  9624 raft_consensus.cc:515] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.794400  9624 leader_election.cc:290] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.791412  9589 raft_consensus.cc:385] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.795181  9589 raft_consensus.cc:740] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.802695  9589 consensus_queue.cc:260] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.806888  9590 tablet_bootstrap.cc:492] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.807384  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157120855af2424da85e64e0c5d847a0" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.808234  9470 raft_consensus.cc:2468] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 0.
I20260709 12:08:17.807427  9590 ts_tablet_manager.cc:1403] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.006s
I20260709 12:08:17.809394  9589 ts_tablet_manager.cc:1434] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.011s
I20260709 12:08:17.810770  9589 tablet_bootstrap.cc:492] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.810818  9362 leader_election.cc:304] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:17.790098  9588 tablet_bootstrap.cc:654] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.826287  9590 raft_consensus.cc:359] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.827082  9590 raft_consensus.cc:385] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.827378  9590 raft_consensus.cc:740] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.828105  9590 consensus_queue.cc:260] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.824285  9624 raft_consensus.cc:2804] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.816735  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157120855af2424da85e64e0c5d847a0" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.833058  9624 raft_consensus.cc:493] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.833808  9624 raft_consensus.cc:3060] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:08:17.834930  9359 leader_election.cc:343] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e494877066cb40059085eceafc819c88 (127.9.11.67:39087): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.840035  9590 ts_tablet_manager.cc:1434] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 12:08:17.841184  9590 tablet_bootstrap.cc:492] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4: Bootstrap starting.
I20260709 12:08:17.845947  9290 catalog_manager.cc:5697] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 0 to 1, leader changed from <none> to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:17.855068  9589 tablet_bootstrap.cc:654] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.856235  9590 tablet_bootstrap.cc:654] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.862969  9588 tablet_bootstrap.cc:492] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88: No bootstrap required, opened a new log
I20260709 12:08:17.863508  9588 ts_tablet_manager.cc:1403] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.008s
I20260709 12:08:17.866658  9588 raft_consensus.cc:359] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.867621  9588 raft_consensus.cc:385] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.867998  9588 raft_consensus.cc:740] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.868777  9588 consensus_queue.cc:260] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.871229  9588 ts_tablet_manager.cc:1434] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:08:17.873701  9625 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.874400  9625 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.879698  9625 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.880009  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.883198  9589 tablet_bootstrap.cc:492] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.883818  9589 ts_tablet_manager.cc:1403] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.005s
I20260709 12:08:17.887013  9624 raft_consensus.cc:515] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.889420  9624 leader_election.cc:290] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.891424  9589 raft_consensus.cc:359] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
W20260709 12:08:17.892072  9362 leader_election.cc:343] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:17.892215  9589 raft_consensus.cc:385] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.892575  9589 raft_consensus.cc:740] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.886291  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.893241  9544 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 0.
I20260709 12:08:17.893427  9589 consensus_queue.cc:260] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.894596  9359 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:17.895583  9624 raft_consensus.cc:2804] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.895807  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157120855af2424da85e64e0c5d847a0" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:17.896029  9624 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.896427  9544 raft_consensus.cc:3060] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.896481  9624 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.900014  9596 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.900687  9596 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.908197  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:17.909942  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157120855af2424da85e64e0c5d847a0" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:17.910753  9470 raft_consensus.cc:3060] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.913769  9589 ts_tablet_manager.cc:1434] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.005s
I20260709 12:08:17.914928  9589 tablet_bootstrap.cc:492] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67: Bootstrap starting.
I20260709 12:08:17.915457  9590 tablet_bootstrap.cc:492] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4: No bootstrap required, opened a new log
I20260709 12:08:17.916076  9590 ts_tablet_manager.cc:1403] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.017s
I20260709 12:08:17.905143  9290 catalog_manager.cc:5697] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:17.918082  9290 catalog_manager.cc:5697] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:17.928560  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.926853  9596 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.929562  9545 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 0.
I20260709 12:08:17.931444  9434 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:17.932775  9596 raft_consensus.cc:2804] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.933203  9596 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.933605  9596 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.933729  9590 raft_consensus.cc:359] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.934497  9590 raft_consensus.cc:385] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.934820  9590 raft_consensus.cc:740] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.935477  9589 tablet_bootstrap.cc:654] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:17.935761  9590 consensus_queue.cc:260] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.946317  9589 tablet_bootstrap.cc:492] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67: No bootstrap required, opened a new log
I20260709 12:08:17.946923  9589 ts_tablet_manager.cc:1403] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.010s
I20260709 12:08:17.948232  9590 ts_tablet_manager.cc:1434] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.003s
I20260709 12:08:17.951134  9589 raft_consensus.cc:359] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.951860  9589 raft_consensus.cc:385] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:17.952128  9589 raft_consensus.cc:740] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Initialized, Role: FOLLOWER
I20260709 12:08:17.953009  9589 consensus_queue.cc:260] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.961795  9589 ts_tablet_manager.cc:1434] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 12:08:17.962349  9620 raft_consensus.cc:493] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.963171  9620 raft_consensus.cc:515] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.967759  9646 raft_consensus.cc:493] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.968648  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:17.968267  9622 consensus_queue.cc:1048] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:17.968922  9646 raft_consensus.cc:515] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:17.969648  9396 raft_consensus.cc:2468] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 0.
I20260709 12:08:17.972494  9508 leader_election.cc:304] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:17.974113  9646 leader_election.cc:290] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:17.975167  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:17.975677  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:17.976922  9396 raft_consensus.cc:2468] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 0.
I20260709 12:08:17.975813  9622 raft_consensus.cc:2804] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.980989  9545 raft_consensus.cc:2468] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 0.
I20260709 12:08:17.981617  9622 raft_consensus.cc:493] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.981388  9434 leader_election.cc:304] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:17.982124  9622 raft_consensus.cc:3060] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:17.983050  9620 leader_election.cc:290] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:17.983721  9646 raft_consensus.cc:2804] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:17.984602  9471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67" is_pre_election: true
I20260709 12:08:17.985689  9646 raft_consensus.cc:493] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.987257  9646 raft_consensus.cc:3060] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.001720  9620 consensus_queue.cc:1048] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.006166  9470 raft_consensus.cc:2468] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:18.006378  9596 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.007809  9362 leader_election.cc:304] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:18.008788  9625 raft_consensus.cc:2804] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:18.009310  9625 raft_consensus.cc:697] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:18.010255  9625 consensus_queue.cc:237] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.011180  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:18.011861  9545 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.006166  9544 raft_consensus.cc:2468] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 1.
I20260709 12:08:18.008930  9596 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.013916  9624 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.016310  9624 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.018494  9394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:18.020095  9395 raft_consensus.cc:2393] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f13e6361ca548a08c36d273c6084d67 in current term 1: Already voted for candidate 9881f5442a5f43c689c38301833d86c4 in this term.
I20260709 12:08:18.022755  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:18.014345  9596 raft_consensus.cc:493] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:18.024097  9470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:18.025228  9470 raft_consensus.cc:2393] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9881f5442a5f43c689c38301833d86c4 in current term 1: Already voted for candidate 7f13e6361ca548a08c36d273c6084d67 in this term.
I20260709 12:08:18.027084  9362 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4; no voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88
I20260709 12:08:18.028505  9624 raft_consensus.cc:2749] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:08:18.024571  9596 raft_consensus.cc:515] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.038800  9596 leader_election.cc:290] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.039474  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dc54a439bef47fe9318f4f1fa6edcfd" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:18.040199  9396 raft_consensus.cc:2468] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 0.
I20260709 12:08:18.040786  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dc54a439bef47fe9318f4f1fa6edcfd" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:18.041435  9544 raft_consensus.cc:2468] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 0.
I20260709 12:08:18.042836  9434 leader_election.cc:304] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:18.046384  9596 raft_consensus.cc:2804] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:18.047320  9596 raft_consensus.cc:493] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:18.048856  9596 raft_consensus.cc:3060] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.053833  9290 catalog_manager.cc:5697] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.064608  9647 raft_consensus.cc:493] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:18.065171  9647 raft_consensus.cc:515] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.067440  9647 leader_election.cc:290] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.068977  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4" is_pre_election: true
I20260709 12:08:18.069639  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88" is_pre_election: true
I20260709 12:08:18.070533  9544 raft_consensus.cc:2393] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f13e6361ca548a08c36d273c6084d67 in current term 1: Already voted for candidate 9881f5442a5f43c689c38301833d86c4 in this term.
I20260709 12:08:18.072201  9434 leader_election.cc:304] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67; no voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88
I20260709 12:08:18.073019  9647 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.086613  9625 consensus_queue.cc:1048] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.087729  9624 consensus_queue.cc:1048] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.096612  9624 consensus_queue.cc:1048] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.111763  9646 raft_consensus.cc:515] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.112255  9622 raft_consensus.cc:515] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.113453  9471 raft_consensus.cc:2393] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e494877066cb40059085eceafc819c88 in current term 1: Already voted for candidate 7f13e6361ca548a08c36d273c6084d67 in this term.
I20260709 12:08:18.114449  9622 leader_election.cc:290] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:18.114494  9646 leader_election.cc:290] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.116688  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:18.116848  9543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:18.117300  9396 raft_consensus.cc:3060] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.117730  9543 raft_consensus.cc:2393] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7f13e6361ca548a08c36d273c6084d67 in current term 1: Already voted for candidate e494877066cb40059085eceafc819c88 in this term.
I20260709 12:08:18.119426  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:18.121666  9434 leader_election.cc:304] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67; no voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88
I20260709 12:08:18.111691  9545 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 1.
I20260709 12:08:18.124094  9646 raft_consensus.cc:2749] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:08:18.126084  9471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56139526e7f64ab582bf591daa967028" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:18.127020  9471 raft_consensus.cc:2393] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e494877066cb40059085eceafc819c88 in current term 1: Already voted for candidate 7f13e6361ca548a08c36d273c6084d67 in this term.
I20260709 12:08:18.123521  9434 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:18.136513  9646 raft_consensus.cc:2804] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:18.137086  9646 raft_consensus.cc:697] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:18.137993  9646 consensus_queue.cc:237] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.141945  9622 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.145690  9625 consensus_queue.cc:1048] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.193596  9622 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.208613  9396 raft_consensus.cc:2468] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 1.
I20260709 12:08:18.213356  9596 raft_consensus.cc:515] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.220374  9596 leader_election.cc:290] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.222424  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dc54a439bef47fe9318f4f1fa6edcfd" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:18.223060  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dc54a439bef47fe9318f4f1fa6edcfd" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:18.223385  9396 raft_consensus.cc:3060] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.224064  9544 raft_consensus.cc:3060] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:18.216364  9508 leader_election.cc:304] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:18.228646  9622 raft_consensus.cc:2804] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:18.230237  9622 raft_consensus.cc:697] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:18.231927  9622 consensus_queue.cc:237] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.234894  9617 consensus_queue.cc:1048] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.271239  9292 catalog_manager.cc:5697] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.276186  9290 catalog_manager.cc:5697] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 0 to 1, leader changed from <none> to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:18.307057  9544 raft_consensus.cc:2468] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 1.
I20260709 12:08:18.307420  9396 raft_consensus.cc:2468] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 1.
I20260709 12:08:18.307114  9647 raft_consensus.cc:2749] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:18.308705  9434 leader_election.cc:304] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:18.310513  9647 raft_consensus.cc:2804] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:18.311041  9647 raft_consensus.cc:697] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:18.311942  9647 consensus_queue.cc:237] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.315045  9617 consensus_queue.cc:1048] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.324770  9290 catalog_manager.cc:5697] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.429176  9617 consensus_queue.cc:1048] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.442929  9617 consensus_queue.cc:1048] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.541237  9647 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.553138  9596 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.604635  9470 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:59942
I20260709 12:08:18.605516  9470 raft_consensus.cc:606] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Received request to transfer leadership to TS 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:18.621832  9396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "157120855af2424da85e64e0c5d847a0"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:08:18.622663  9396 raft_consensus.cc:606] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:18.643886  9544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "e494877066cb40059085eceafc819c88"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:58156
I20260709 12:08:18.644667  9544 raft_consensus.cc:606] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:18.651710  9544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8"
dest_uuid: "e494877066cb40059085eceafc819c88"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:58156
I20260709 12:08:18.652377  9544 raft_consensus.cc:606] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:18.656189  9622 consensus_queue.cc:1048] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.679177  9650 consensus_queue.cc:1048] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.810606  9647 consensus_queue.cc:1048] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.831389  9650 raft_consensus.cc:993] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:18.831529  9647 consensus_queue.cc:1048] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:18.832099  9650 raft_consensus.cc:1081] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:18.835167  9471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59940
I20260709 12:08:18.835826  9471 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.836256  9471 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.846884  9471 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.850957  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:18.850961  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:18.851806  9396 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.851902  9544 raft_consensus.cc:3055] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:18.852305  9544 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:18.853238  9544 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.854697  9544 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.849812  9471 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:18.859961  9396 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:18.861629  9434 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:18.862730  9647 raft_consensus.cc:2804] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:18.863305  9647 raft_consensus.cc:697] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:18.864316  9647 consensus_queue.cc:237] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:18.873937  9544 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:18.874974  9293 catalog_manager.cc:5697] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 1 to 2, leader changed from e494877066cb40059085eceafc819c88 (127.9.11.67) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:19.002545  9646 raft_consensus.cc:993] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:19.003101  9647 raft_consensus.cc:1081] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Signalling peer 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:19.004721  9396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:57766
I20260709 12:08:19.005300  9396 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:19.005683  9396 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.011615  9396 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.013995  9396 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 2 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:19.015568  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:19.015829  9471 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:19.016345  9544 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.016750  9471 raft_consensus.cc:3055] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:19.017328  9471 raft_consensus.cc:740] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:19.018654  9471 consensus_queue.cc:260] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.020759  9471 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.024003  9544 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 2.
I20260709 12:08:19.025723  9359 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:19.027304  9595 raft_consensus.cc:2804] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:19.027958  9595 raft_consensus.cc:697] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:19.028867  9595 consensus_queue.cc:237] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.031669  9471 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 2.
I20260709 12:08:19.038774  9293 catalog_manager.cc:5697] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 1 to 2, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:19.053293  9622 raft_consensus.cc:993] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:19.053866  9650 raft_consensus.cc:1081] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:19.055545  9471 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59940
I20260709 12:08:19.056074  9471 raft_consensus.cc:493] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:19.056397  9471 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.061434  9471 raft_consensus.cc:515] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.063540  9471 leader_election.cc:290] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:19.065469  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:19.065842  9617 raft_consensus.cc:993] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:19.066092  9396 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.066313  9617 raft_consensus.cc:1081] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:19.066326  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:19.067198  9544 raft_consensus.cc:3055] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:19.067566  9544 raft_consensus.cc:740] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:19.068192  9469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "157120855af2424da85e64e0c5d847a0"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59936
I20260709 12:08:19.068715  9469 raft_consensus.cc:493] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:19.069084  9469 raft_consensus.cc:3060] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.072902  9396 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:19.074297  9434 leader_election.cc:304] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:19.074254  9544 consensus_queue.cc:260] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.075416  9647 raft_consensus.cc:2804] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:19.075956  9647 raft_consensus.cc:697] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:19.075837  9544 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.076857  9647 consensus_queue.cc:237] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.075938  9469 raft_consensus.cc:515] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.082626  9469 leader_election.cc:290] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:19.083364  9544 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:19.083630  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157120855af2424da85e64e0c5d847a0" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:19.084311  9396 raft_consensus.cc:3055] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:19.084728  9396 raft_consensus.cc:740] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:19.085659  9396 consensus_queue.cc:260] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.086126  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157120855af2424da85e64e0c5d847a0" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:19.086730  9544 raft_consensus.cc:3060] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.087033  9396 raft_consensus.cc:3060] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:19.089771  9293 catalog_manager.cc:5697] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 1 to 2, leader changed from e494877066cb40059085eceafc819c88 (127.9.11.67) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:19.094010  9544 raft_consensus.cc:2468] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:19.095806  9434 leader_election.cc:304] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:19.096400  9396 raft_consensus.cc:2468] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:19.097164  9647 raft_consensus.cc:2804] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:19.097772  9647 raft_consensus.cc:697] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:19.099116  9647 consensus_queue.cc:237] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:19.110846  9293 catalog_manager.cc:5697] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 1 to 2, leader changed from 9881f5442a5f43c689c38301833d86c4 (127.9.11.65) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:19.398485  9396 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.400022  9647 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.410573  9544 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.412427  9646 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.436837  9469 raft_consensus.cc:1275] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.438314  9617 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.445871  9544 raft_consensus.cc:1275] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.447527  9595 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.509136  9396 raft_consensus.cc:1275] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.510656  9647 consensus_queue.cc:1048] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:19.521296  9544 raft_consensus.cc:1275] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.523691  9690 consensus_queue.cc:1048] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:19.598311  9396 raft_consensus.cc:1275] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.599680  9690 consensus_queue.cc:1048] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.609357  9544 raft_consensus.cc:1275] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.611117  9690 consensus_queue.cc:1048] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:21.688529  9261 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:08:21.689419  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:21.707152  9396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
mode: GRACEFUL
new_leader_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:43378
I20260709 12:08:21.707759  9396 raft_consensus.cc:606] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 LEADER]: Received request to transfer leadership to TS e494877066cb40059085eceafc819c88
I20260709 12:08:21.718586  9469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:45146
I20260709 12:08:21.719223  9469 raft_consensus.cc:606] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Received request to transfer leadership to TS e494877066cb40059085eceafc819c88
I20260709 12:08:21.720404  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:08:21.720957  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:21.820647  9700 raft_consensus.cc:993] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:21.821249  9690 raft_consensus.cc:1081] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Signalling peer e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:21.823419  9544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:58138
I20260709 12:08:21.824119  9544 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:21.824540  9544 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.830013  9544 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:21.831749  9544 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 3 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:21.833071  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:21.833077  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:21.833905  9469 raft_consensus.cc:3055] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:21.833922  9396 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.834393  9469 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:21.835326  9469 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:21.836632  9469 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.841879  9396 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 3.
I20260709 12:08:21.843398  9508 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:21.844063  9469 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 3.
I20260709 12:08:21.844303  9650 raft_consensus.cc:2804] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:21.846107  9650 raft_consensus.cc:697] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:21.847012  9650 consensus_queue.cc:237] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:21.856128  9291 catalog_manager.cc:5697] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 2 to 3, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:21.858702  9695 raft_consensus.cc:993] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: : Instructing follower e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:21.859354  9703 raft_consensus.cc:1081] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 LEADER]: Signalling peer e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:21.861567  9544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6"
dest_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:58146
I20260709 12:08:21.862164  9544 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:21.862484  9544 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.869452  9544 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:21.872337  9396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:21.872521  9544 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 3 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:21.873055  9396 raft_consensus.cc:3055] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:21.873116  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:21.873587  9396 raft_consensus.cc:740] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:21.873805  9469 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.874488  9396 consensus_queue.cc:260] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:21.875661  9396 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.880231  9469 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 3.
I20260709 12:08:21.881475  9396 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 3.
I20260709 12:08:21.881412  9511 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:21.882413  9650 raft_consensus.cc:2804] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:21.882893  9650 raft_consensus.cc:697] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:21.883776  9650 consensus_queue.cc:237] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:21.890942  9291 catalog_manager.cc:5697] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 2 to 3, leader changed from 9881f5442a5f43c689c38301833d86c4 (127.9.11.65) to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:22.274030  9396 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.275694  9650 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:22.286958  9469 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.289445  9701 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:22.367772  9469 raft_consensus.cc:1275] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.369379  9701 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:22.382689  9396 raft_consensus.cc:1275] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.385612  9715 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:22.722237  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:22.728446  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:22.728937  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:23.730129  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:23.735760  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:23.736282  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:24.737833  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:24.744248  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:24.744781  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:25.745921  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:25.751260  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:25.751847  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:26.752871  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:26.758030  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:26.758541  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:27.759759  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:27.765353  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:27.765834  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:28.766824  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:28.772106  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:28.772518  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:29.773434  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:29.779099  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:29.779529  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:30.780614  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:30.786159  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:30.786577  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:31.815161  9261 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:08:31.820919  9395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bea185737654ec19e81900b0fb53191"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:08:31.821941  9395 raft_consensus.cc:606] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:31.827966  9395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:08:31.828572  9395 raft_consensus.cc:606] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:31.833280  9395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:57792
I20260709 12:08:31.833777  9395 raft_consensus.cc:606] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:31.838310  9544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6"
dest_uuid: "e494877066cb40059085eceafc819c88"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:58156
I20260709 12:08:31.838809  9544 raft_consensus.cc:606] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:31.848083  9544 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "e494877066cb40059085eceafc819c88"
mode: GRACEFUL
new_leader_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:58156
I20260709 12:08:31.848588  9544 raft_consensus.cc:606] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Received request to transfer leadership to TS 7f13e6361ca548a08c36d273c6084d67
I20260709 12:08:31.888581  9720 raft_consensus.cc:993] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:31.889195  9732 raft_consensus.cc:1081] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:31.890838  9469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bea185737654ec19e81900b0fb53191"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59936
I20260709 12:08:31.891479  9469 raft_consensus.cc:493] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.891848  9469 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.897572  9469 raft_consensus.cc:515] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:31.899230  9469 leader_election.cc:290] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Requested vote from peers e494877066cb40059085eceafc819c88 (127.9.11.67:39087), 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381)
I20260709 12:08:31.900272  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:31.900396  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:31.900974  9544 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.901005  9395 raft_consensus.cc:3055] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:31.901423  9395 raft_consensus.cc:740] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:31.902215  9395 consensus_queue.cc:260] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:31.903321  9395 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.906159  9544 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:31.907392  9434 leader_election.cc:304] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:31.908224  9728 raft_consensus.cc:2804] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:31.909361  9728 raft_consensus.cc:697] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:31.909586  9395 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:31.910630  9728 consensus_queue.cc:237] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:31.917490  9735 raft_consensus.cc:993] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:31.918109  9729 raft_consensus.cc:1081] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:31.919590  9469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59940
I20260709 12:08:31.920097  9469 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.920398  9469 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.920410  9293 catalog_manager.cc:5697] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 1 to 2, leader changed from 9881f5442a5f43c689c38301833d86c4 (127.9.11.65) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:31.927827  9469 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:31.930305  9469 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 4 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:31.931669  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:31.932065  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:31.932504  9395 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.932698  9544 raft_consensus.cc:3055] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:08:31.933109  9544 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:31.934064  9544 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:31.935611  9544 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.940755  9395 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 4.
I20260709 12:08:31.941892  9434 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:31.942380  9544 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 4.
I20260709 12:08:31.942711  9728 raft_consensus.cc:2804] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:08:31.943248  9728 raft_consensus.cc:697] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:31.944193  9728 consensus_queue.cc:237] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:31.951479  9293 catalog_manager.cc:5697] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 3 to 4, leader changed from e494877066cb40059085eceafc819c88 (127.9.11.67) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:32.074292  9720 raft_consensus.cc:993] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:32.074800  9732 raft_consensus.cc:1081] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:32.076260  9469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59936
I20260709 12:08:32.076745  9469 raft_consensus.cc:493] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:32.077016  9469 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:32.082806  9469 raft_consensus.cc:515] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.084293  9469 leader_election.cc:290] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:32.085052  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:32.085340  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:32.085614  9395 raft_consensus.cc:3055] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:32.085889  9544 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:32.085994  9395 raft_consensus.cc:740] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:32.086812  9395 consensus_queue.cc:260] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.087883  9395 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:32.090797  9544 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:32.091799  9434 leader_election.cc:304] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:32.092590  9728 raft_consensus.cc:2804] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:32.093031  9728 raft_consensus.cc:697] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:32.093305  9395 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:32.093945  9728 consensus_queue.cc:237] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.101212  9293 catalog_manager.cc:5697] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 1 to 2, leader changed from 9881f5442a5f43c689c38301833d86c4 (127.9.11.65) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:32.257452  9720 raft_consensus.cc:993] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:32.258014  9732 raft_consensus.cc:1081] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:32.259800  9469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59936
I20260709 12:08:32.260314  9469 raft_consensus.cc:493] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:32.260628  9469 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:32.266414  9469 raft_consensus.cc:515] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.267987  9469 leader_election.cc:290] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:32.269025  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:32.269234  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:32.269766  9544 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:32.269702  9395 raft_consensus.cc:3055] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:32.270238  9395 raft_consensus.cc:740] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:32.271170  9395 consensus_queue.cc:260] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.272501  9395 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:32.277222  9544 raft_consensus.cc:2468] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:32.278486  9434 leader_election.cc:304] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:32.279337  9728 raft_consensus.cc:2804] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:32.279716  9395 raft_consensus.cc:2468] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 2.
I20260709 12:08:32.279779  9728 raft_consensus.cc:697] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:32.280861  9728 consensus_queue.cc:237] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.289110  9293 catalog_manager.cc:5697] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 1 to 2, leader changed from 9881f5442a5f43c689c38301833d86c4 (127.9.11.65) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:32.305012  9395 raft_consensus.cc:1275] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.306320  9728 consensus_queue.cc:1048] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:32.317266  9544 raft_consensus.cc:1275] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.319686  9728 consensus_queue.cc:1048] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:32.353910  9735 raft_consensus.cc:993] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: : Instructing follower 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:32.354415  9729 raft_consensus.cc:1081] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Signalling peer 7f13e6361ca548a08c36d273c6084d67 to start an election
I20260709 12:08:32.356005  9469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
 from {username='slave'} at 127.0.0.1:59940
I20260709 12:08:32.356458  9469 raft_consensus.cc:493] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:32.356735  9469 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:32.361276  9469 raft_consensus.cc:515] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.362851  9469 leader_election.cc:290] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 4 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:32.363936  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:32.364125  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b6f457c9cc43139072ace0befcd7c6" candidate_uuid: "7f13e6361ca548a08c36d273c6084d67" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:32.364814  9544 raft_consensus.cc:3055] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:08:32.364787  9395 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:32.365283  9544 raft_consensus.cc:740] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:32.366125  9544 consensus_queue.cc:260] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.367415  9544 raft_consensus.cc:3060] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:32.372545  9395 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 4.
I20260709 12:08:32.373725  9434 leader_election.cc:304] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f13e6361ca548a08c36d273c6084d67, 9881f5442a5f43c689c38301833d86c4; no voters: 
I20260709 12:08:32.374195  9544 raft_consensus.cc:2468] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f13e6361ca548a08c36d273c6084d67 in term 4.
I20260709 12:08:32.374612  9745 raft_consensus.cc:2804] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:08:32.375164  9745 raft_consensus.cc:697] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Becoming Leader. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:32.376048  9745 consensus_queue.cc:237] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:32.383528  9291 catalog_manager.cc:5697] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 reported cstate change: term changed from 3 to 4, leader changed from e494877066cb40059085eceafc819c88 (127.9.11.67) to 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "7f13e6361ca548a08c36d273c6084d67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:32.483180  9395 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.484586  9745 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:08:32.494410  9544 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.496572  9728 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:08:32.686339  9544 raft_consensus.cc:1275] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.687647  9728 consensus_queue.cc:1048] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:32.696017  9395 raft_consensus.cc:1275] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.697706  9745 consensus_queue.cc:1048] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:32.769115  9544 raft_consensus.cc:1275] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.770274  9745 consensus_queue.cc:1048] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:32.773656  9545 raft_consensus.cc:1275] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.775202  9733 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:08:32.783473  9395 raft_consensus.cc:1275] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.785799  9733 consensus_queue.cc:1048] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:32.792188  9396 raft_consensus.cc:1275] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Refusing update from remote peer 7f13e6361ca548a08c36d273c6084d67: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.794080  9756 consensus_queue.cc:1048] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:08:34.883993  9261 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:08:34.885067  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:34.892679  9469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:45146
I20260709 12:08:34.893465  9469 raft_consensus.cc:606] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Received request to transfer leadership to TS e494877066cb40059085eceafc819c88
I20260709 12:08:34.896145  9469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bea185737654ec19e81900b0fb53191"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:45146
I20260709 12:08:34.896644  9469 raft_consensus.cc:606] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Received request to transfer leadership to TS 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:34.898838  9469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:45146
I20260709 12:08:34.899279  9469 raft_consensus.cc:606] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Received request to transfer leadership to TS 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:34.901413  9469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:45146
I20260709 12:08:34.901858  9469 raft_consensus.cc:606] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Received request to transfer leadership to TS 9881f5442a5f43c689c38301833d86c4
I20260709 12:08:34.903914  9469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8"
dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
mode: GRACEFUL
new_leader_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:45146
I20260709 12:08:34.904328  9469 raft_consensus.cc:606] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Received request to transfer leadership to TS e494877066cb40059085eceafc819c88
I20260709 12:08:34.905321  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:08:34.905808  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:34.933539  9759 raft_consensus.cc:993] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:34.934171  9756 raft_consensus.cc:1081] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Signalling peer 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:34.935652  9395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:57766
I20260709 12:08:34.936101  9395 raft_consensus.cc:493] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:34.936374  9395 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.942516  9395 raft_consensus.cc:515] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:34.945247  9395 leader_election.cc:290] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 3 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:34.946180  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:34.946677  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dcb500384384968ae7f939bef8cdbfc" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:34.947060  9469 raft_consensus.cc:3055] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:34.947369  9545 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.947398  9469 raft_consensus.cc:740] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:34.948231  9469 consensus_queue.cc:260] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:34.949321  9469 raft_consensus.cc:3060] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.953004  9545 raft_consensus.cc:2468] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 3.
I20260709 12:08:34.954309  9359 leader_election.cc:304] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:34.954418  9469 raft_consensus.cc:2468] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 3.
I20260709 12:08:34.956220  9761 raft_consensus.cc:2804] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:34.957119  9761 raft_consensus.cc:697] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 3 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:34.958149  9761 consensus_queue.cc:237] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:34.965864  9293 catalog_manager.cc:5697] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 2 to 3, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:35.110198  9759 raft_consensus.cc:993] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:35.110716  9756 raft_consensus.cc:1081] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Signalling peer 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:35.112226  9395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:57766
I20260709 12:08:35.112748  9395 raft_consensus.cc:493] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:35.113147  9395 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.118671  9395 raft_consensus.cc:515] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.120239  9395 leader_election.cc:290] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 3 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:35.121133  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:35.121300  9545 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5da7e262d3343c7917047b2c07bb630" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:35.121748  9469 raft_consensus.cc:3055] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:35.121817  9545 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.122139  9469 raft_consensus.cc:740] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:35.123040  9469 consensus_queue.cc:260] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.124297  9469 raft_consensus.cc:3060] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.125911  9759 raft_consensus.cc:993] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:35.126399  9756 raft_consensus.cc:1081] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Signalling peer e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:35.128048  9544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8"
dest_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:58138
I20260709 12:08:35.128576  9544 raft_consensus.cc:493] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:35.128867  9544 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.129376  9545 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 3.
I20260709 12:08:35.130782  9359 leader_election.cc:304] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:35.131630  9761 raft_consensus.cc:2804] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:35.131877  9469 raft_consensus.cc:2468] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 3.
I20260709 12:08:35.132170  9761 raft_consensus.cc:697] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 3 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:35.133188  9761 consensus_queue.cc:237] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.135847  9544 raft_consensus.cc:515] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.139479  9544 leader_election.cc:290] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 3 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:35.139607  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:35.139917  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f689a6b646c5419cb6432592e047a1b8" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:35.140259  9395 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.140509  9469 raft_consensus.cc:3055] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:35.140837  9469 raft_consensus.cc:740] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:35.141590  9469 consensus_queue.cc:260] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.142354  9293 catalog_manager.cc:5697] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 2 to 3, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:35.142796  9469 raft_consensus.cc:3060] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.149273  9395 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 3.
I20260709 12:08:35.150094  9469 raft_consensus.cc:2468] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 3.
I20260709 12:08:35.150816  9508 leader_election.cc:304] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:35.151837  9729 raft_consensus.cc:2804] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:35.153234  9729 raft_consensus.cc:697] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:35.154111  9729 consensus_queue.cc:237] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.162194  9293 catalog_manager.cc:5697] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 2 to 3, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:35.187142  9759 raft_consensus.cc:993] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:35.187625  9756 raft_consensus.cc:1081] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Signalling peer e494877066cb40059085eceafc819c88 to start an election
I20260709 12:08:35.189092  9544 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41bad849598b42a283ba311a077d4c41"
dest_uuid: "e494877066cb40059085eceafc819c88"
 from {username='slave'} at 127.0.0.1:58138
I20260709 12:08:35.189546  9544 raft_consensus.cc:493] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:35.189800  9544 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:08:35.194523  9544 raft_consensus.cc:515] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.196164  9544 leader_election.cc:290] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 5 election: Requested vote from peers 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381), 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955)
I20260709 12:08:35.197176  9395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9881f5442a5f43c689c38301833d86c4"
I20260709 12:08:35.197310  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bad849598b42a283ba311a077d4c41" candidate_uuid: "e494877066cb40059085eceafc819c88" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:35.197824  9395 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:08:35.197842  9469 raft_consensus.cc:3055] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:08:35.198342  9469 raft_consensus.cc:740] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:35.199154  9469 consensus_queue.cc:260] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.200467  9469 raft_consensus.cc:3060] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:08:35.204900  9395 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 5.
I20260709 12:08:35.206077  9508 leader_election.cc:304] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:35.206802  9729 raft_consensus.cc:2804] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:08:35.206861  9469 raft_consensus.cc:2468] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e494877066cb40059085eceafc819c88 in term 5.
I20260709 12:08:35.207412  9729 raft_consensus.cc:697] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 5 LEADER]: Becoming Leader. State: Replica: e494877066cb40059085eceafc819c88, State: Running, Role: LEADER
I20260709 12:08:35.208282  9729 consensus_queue.cc:237] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } }
I20260709 12:08:35.217347  9293 catalog_manager.cc:5697] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 reported cstate change: term changed from 4 to 5, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to e494877066cb40059085eceafc819c88 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "e494877066cb40059085eceafc819c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:35.297780  9756 raft_consensus.cc:993] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: : Instructing follower 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:35.298269  9759 raft_consensus.cc:1081] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Signalling peer 9881f5442a5f43c689c38301833d86c4 to start an election
I20260709 12:08:35.299894  9395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bea185737654ec19e81900b0fb53191"
dest_uuid: "9881f5442a5f43c689c38301833d86c4"
 from {username='slave'} at 127.0.0.1:57766
I20260709 12:08:35.300469  9395 raft_consensus.cc:493] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:35.300815  9395 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.305693  9395 raft_consensus.cc:515] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:35.307228  9395 leader_election.cc:290] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 3 election: Requested vote from peers 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955), e494877066cb40059085eceafc819c88 (127.9.11.67:39087)
I20260709 12:08:35.308132  9469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7f13e6361ca548a08c36d273c6084d67"
I20260709 12:08:35.308247  9544 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bea185737654ec19e81900b0fb53191" candidate_uuid: "9881f5442a5f43c689c38301833d86c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e494877066cb40059085eceafc819c88"
I20260709 12:08:35.308717  9469 raft_consensus.cc:3055] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:35.308753  9544 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.309041  9469 raft_consensus.cc:740] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7f13e6361ca548a08c36d273c6084d67, State: Running, Role: LEADER
I20260709 12:08:35.309687  9469 consensus_queue.cc:260] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:35.310612  9469 raft_consensus.cc:3060] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.313604  9544 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 3.
I20260709 12:08:35.314589  9359 leader_election.cc:304] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9881f5442a5f43c689c38301833d86c4, e494877066cb40059085eceafc819c88; no voters: 
I20260709 12:08:35.315348  9761 raft_consensus.cc:2804] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:35.315454  9469 raft_consensus.cc:2468] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9881f5442a5f43c689c38301833d86c4 in term 3.
I20260709 12:08:35.315922  9761 raft_consensus.cc:697] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 3 LEADER]: Becoming Leader. State: Replica: 9881f5442a5f43c689c38301833d86c4, State: Running, Role: LEADER
I20260709 12:08:35.316956  9761 consensus_queue.cc:237] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } }
I20260709 12:08:35.324945  9293 catalog_manager.cc:5697] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 reported cstate change: term changed from 2 to 3, leader changed from 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66) to 9881f5442a5f43c689c38301833d86c4 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "9881f5442a5f43c689c38301833d86c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:35.346704  9544 raft_consensus.cc:1275] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.348002  9761 consensus_queue.cc:1048] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.360821  9469 raft_consensus.cc:1275] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.362278  9762 consensus_queue.cc:1048] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:35.648623  9469 raft_consensus.cc:1275] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.649880  9762 consensus_queue.cc:1048] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.656095  9395 raft_consensus.cc:1275] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.657738  9729 consensus_queue.cc:1048] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.662293  9471 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 5 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:08:35.663671  9544 raft_consensus.cc:1275] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.664209  9769 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:08:35.668329  9762 consensus_queue.cc:1048] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.688758  9469 raft_consensus.cc:1275] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.691725  9783 consensus_queue.cc:1048] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:08:35.695894  9395 raft_consensus.cc:1275] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 5 FOLLOWER]: Refusing update from remote peer e494877066cb40059085eceafc819c88: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:08:35.697942  9769 consensus_queue.cc:1048] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9881f5442a5f43c689c38301833d86c4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:08:35.710528  9471 raft_consensus.cc:1275] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.714036  9762 consensus_queue.cc:1048] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f13e6361ca548a08c36d273c6084d67" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.723614  9544 raft_consensus.cc:1275] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Refusing update from remote peer 9881f5442a5f43c689c38301833d86c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.725571  9779 consensus_queue.cc:1048] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e494877066cb40059085eceafc819c88" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:35.907311  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:35.912554  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:35.913098  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:36.914429  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:36.919790  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:36.920305  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:37.921487  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:37.926818  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:37.927278  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:38.928552  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:38.934826  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:38.935420  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:39.936610  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:39.942873  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:39.943320  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:40.944416  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:40.949970  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:40.950552  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:41.951861  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:41.957394  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:41.957914  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:42.959136  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:42.964707  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:42.965198  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:43.966423  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:43.972280  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:43.972821  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:45.004220  9261 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:08:45.005079  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
I20260709 12:08:45.041039  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:08:45.042085  9261 tablet_replica.cc:333] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.043103  9261 raft_consensus.cc:2243] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.043908  9261 raft_consensus.cc:2272] T 4dc54a439bef47fe9318f4f1fa6edcfd P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.047254  9261 tablet_replica.cc:333] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.047900  9261 raft_consensus.cc:2243] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.048451  9261 raft_consensus.cc:2272] T 56139526e7f64ab582bf591daa967028 P 9881f5442a5f43c689c38301833d86c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.050949  9261 tablet_replica.cc:333] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.051568  9261 raft_consensus.cc:2243] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.052027  9261 raft_consensus.cc:2272] T 12b6f457c9cc43139072ace0befcd7c6 P 9881f5442a5f43c689c38301833d86c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.054155  9261 tablet_replica.cc:333] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.054636  9261 raft_consensus.cc:2243] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.055069  9261 raft_consensus.cc:2272] T 157120855af2424da85e64e0c5d847a0 P 9881f5442a5f43c689c38301833d86c4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.056814  9261 tablet_replica.cc:333] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.057307  9261 raft_consensus.cc:2243] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.058163  9261 raft_consensus.cc:2272] T a5da7e262d3343c7917047b2c07bb630 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.060226  9261 tablet_replica.cc:333] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.060954  9261 raft_consensus.cc:2243] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.061918  9261 raft_consensus.cc:2272] T 9dcb500384384968ae7f939bef8cdbfc P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:08:45.062368  9508 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:08:45.065042  9261 tablet_replica.cc:333] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.066061  9261 raft_consensus.cc:2243] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:08:45.066844  9508 consensus_peers.cc:597] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 -> Peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Couldn't send request to peer 9881f5442a5f43c689c38301833d86c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.067498  9261 raft_consensus.cc:2272] T 8bea185737654ec19e81900b0fb53191 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.070585  9261 tablet_replica.cc:333] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.071445  9261 raft_consensus.cc:2243] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.072161  9261 raft_consensus.cc:2272] T 41bad849598b42a283ba311a077d4c41 P 9881f5442a5f43c689c38301833d86c4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.074760  9261 tablet_replica.cc:333] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4: stopping tablet replica
I20260709 12:08:45.075516  9261 raft_consensus.cc:2243] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.076108  9261 raft_consensus.cc:2272] T f689a6b646c5419cb6432592e047a1b8 P 9881f5442a5f43c689c38301833d86c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.193714  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
W20260709 12:08:45.195178  9434 consensus_peers.cc:597] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 -> Peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Couldn't send request to peer 9881f5442a5f43c689c38301833d86c4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:35381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.219580  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
I20260709 12:08:45.254685  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:08:45.255681  9261 tablet_replica.cc:333] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.256831  9261 raft_consensus.cc:2243] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.257740  9261 raft_consensus.cc:2272] T 56139526e7f64ab582bf591daa967028 P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.259912  9261 tablet_replica.cc:333] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.260458  9261 raft_consensus.cc:2243] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.261394  9261 raft_consensus.cc:2272] T 12b6f457c9cc43139072ace0befcd7c6 P 7f13e6361ca548a08c36d273c6084d67 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.263353  9261 tablet_replica.cc:333] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.263875  9261 raft_consensus.cc:2243] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.264665  9261 raft_consensus.cc:2272] T 4dc54a439bef47fe9318f4f1fa6edcfd P 7f13e6361ca548a08c36d273c6084d67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.266445  9261 tablet_replica.cc:333] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.266921  9261 raft_consensus.cc:2243] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.267843  9261 raft_consensus.cc:2272] T 157120855af2424da85e64e0c5d847a0 P 7f13e6361ca548a08c36d273c6084d67 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.269549  9261 tablet_replica.cc:333] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.270035  9261 raft_consensus.cc:2243] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.270524  9261 raft_consensus.cc:2272] T a5da7e262d3343c7917047b2c07bb630 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.272276  9261 tablet_replica.cc:333] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.272811  9261 raft_consensus.cc:2243] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.273253  9261 raft_consensus.cc:2272] T 9dcb500384384968ae7f939bef8cdbfc P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.274941  9261 tablet_replica.cc:333] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.275512  9261 raft_consensus.cc:2243] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.275978  9261 raft_consensus.cc:2272] T 41bad849598b42a283ba311a077d4c41 P 7f13e6361ca548a08c36d273c6084d67 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.277779  9261 tablet_replica.cc:333] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.278267  9261 raft_consensus.cc:2243] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.278712  9261 raft_consensus.cc:2272] T 8bea185737654ec19e81900b0fb53191 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.280544  9261 tablet_replica.cc:333] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67: stopping tablet replica
I20260709 12:08:45.281013  9261 raft_consensus.cc:2243] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.281461  9261 raft_consensus.cc:2272] T f689a6b646c5419cb6432592e047a1b8 P 7f13e6361ca548a08c36d273c6084d67 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.313496  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
W20260709 12:08:45.321213  9511 consensus_peers.cc:597] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 -> Peer 7f13e6361ca548a08c36d273c6084d67 (127.9.11.66:32955): Couldn't send request to peer 7f13e6361ca548a08c36d273c6084d67. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:32955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.337950  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
W20260709 12:08:45.357928  9508 consensus_peers.cc:597] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 -> Peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Couldn't send request to peer 9881f5442a5f43c689c38301833d86c4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:35381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.369494  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:08:45.370550  9261 tablet_replica.cc:333] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.371371  9261 raft_consensus.cc:2243] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.372430  9261 raft_consensus.cc:2272] T 56139526e7f64ab582bf591daa967028 P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.374552  9261 tablet_replica.cc:333] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.375247  9261 raft_consensus.cc:2243] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.375757  9261 raft_consensus.cc:2272] T 157120855af2424da85e64e0c5d847a0 P e494877066cb40059085eceafc819c88 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.377830  9261 tablet_replica.cc:333] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.378592  9261 raft_consensus.cc:2243] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.379276  9261 raft_consensus.cc:2272] T a5da7e262d3343c7917047b2c07bb630 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.381666  9261 tablet_replica.cc:333] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88: stopping tablet replica
W20260709 12:08:45.381829  9508 consensus_peers.cc:597] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 -> Peer 9881f5442a5f43c689c38301833d86c4 (127.9.11.65:35381): Couldn't send request to peer 9881f5442a5f43c689c38301833d86c4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:35381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.382414  9261 raft_consensus.cc:2243] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.383365  9261 raft_consensus.cc:2272] T 41bad849598b42a283ba311a077d4c41 P e494877066cb40059085eceafc819c88 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.385083  9261 tablet_replica.cc:333] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.385578  9261 raft_consensus.cc:2243] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.386018  9261 raft_consensus.cc:2272] T 9dcb500384384968ae7f939bef8cdbfc P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.387874  9261 tablet_replica.cc:333] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.388343  9261 raft_consensus.cc:2243] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.388733  9261 raft_consensus.cc:2272] T 8bea185737654ec19e81900b0fb53191 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.390421  9261 tablet_replica.cc:333] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.390877  9261 raft_consensus.cc:2243] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.391341  9261 raft_consensus.cc:2272] T 4dc54a439bef47fe9318f4f1fa6edcfd P e494877066cb40059085eceafc819c88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.393038  9261 tablet_replica.cc:333] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.393463  9261 raft_consensus.cc:2243] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.394243  9261 raft_consensus.cc:2272] T f689a6b646c5419cb6432592e047a1b8 P e494877066cb40059085eceafc819c88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.396008  9261 tablet_replica.cc:333] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88: stopping tablet replica
I20260709 12:08:45.396426  9261 raft_consensus.cc:2243] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.396788  9261 raft_consensus.cc:2272] T 12b6f457c9cc43139072ace0befcd7c6 P e494877066cb40059085eceafc819c88 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.426620  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:08:45.448601  9261 master.cc:562] Master@127.9.11.126:36783 shutting down...
I20260709 12:08:45.601028  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.601824  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.602200  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P 75a097bc3547474393a99c0686c0ae5b: stopping tablet replica
I20260709 12:08:45.623907  9261 master.cc:584] Master@127.9.11.126:36783 shutdown complete.
7f13e6361ca548a08c36d273c6084d67  4
9881f5442a5f43c689c38301833d86c4  4
e494877066cb40059085eceafc819c88  1
7f13e6361ca548a08c36d273c6084d67  3
9881f5442a5f43c689c38301833d86c4  3
e494877066cb40059085eceafc819c88  3
7f13e6361ca548a08c36d273c6084d67  8
9881f5442a5f43c689c38301833d86c4  0
e494877066cb40059085eceafc819c88  1
7f13e6361ca548a08c36d273c6084d67  3
9881f5442a5f43c689c38301833d86c4  3
e494877066cb40059085eceafc819c88  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31462 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:08:45.671816  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:44893
I20260709 12:08:45.673198  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:45.680053  9803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:45.680534  9805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:45.680066  9802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:45.681269  9261 server_base.cc:1061] running on GCE node
I20260709 12:08:45.682292  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:45.682525  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:45.682699  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598925682673 us; error 0 us; skew 500 ppm
I20260709 12:08:45.683351  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:45.686028  9261 webserver.cc:533] Webserver started at http://127.9.11.126:45421/ using document root <none> and password file <none>
I20260709 12:08:45.686582  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:45.686811  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:45.687165  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:45.688503  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "a383cf8f549e4ebe990468b3cc2bb7c1"
format_stamp: "Formatted at 2026-07-09 12:08:45 on dist-test-slave-k289"
I20260709 12:08:45.693358  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:08:45.697219  9810 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:45.698150  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:08:45.698410  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "a383cf8f549e4ebe990468b3cc2bb7c1"
format_stamp: "Formatted at 2026-07-09 12:08:45 on dist-test-slave-k289"
I20260709 12:08:45.698673  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:45.716005  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:45.717429  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:45.763422  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:44893
I20260709 12:08:45.763556  9861 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:44893 every 8 connection(s)
I20260709 12:08:45.767777  9862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:45.778366  9862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1: Bootstrap starting.
I20260709 12:08:45.783742  9862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:45.788543  9862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1: No bootstrap required, opened a new log
I20260709 12:08:45.791076  9862 raft_consensus.cc:359] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" member_type: VOTER }
I20260709 12:08:45.791594  9862 raft_consensus.cc:385] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:45.791841  9862 raft_consensus.cc:740] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a383cf8f549e4ebe990468b3cc2bb7c1, State: Initialized, Role: FOLLOWER
I20260709 12:08:45.792498  9862 consensus_queue.cc:260] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" member_type: VOTER }
I20260709 12:08:45.793030  9862 raft_consensus.cc:399] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:08:45.793267  9862 raft_consensus.cc:493] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:08:45.793581  9862 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:45.798060  9862 raft_consensus.cc:515] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" member_type: VOTER }
I20260709 12:08:45.798620  9862 leader_election.cc:304] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a383cf8f549e4ebe990468b3cc2bb7c1; no voters: 
I20260709 12:08:45.799932  9862 leader_election.cc:290] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:08:45.800309  9865 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:45.801852  9865 raft_consensus.cc:697] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 1 LEADER]: Becoming Leader. State: Replica: a383cf8f549e4ebe990468b3cc2bb7c1, State: Running, Role: LEADER
I20260709 12:08:45.802637  9865 consensus_queue.cc:237] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" member_type: VOTER }
I20260709 12:08:45.803318  9862 sys_catalog.cc:565] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:08:45.807504  9866 sys_catalog.cc:455] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" member_type: VOTER } }
I20260709 12:08:45.808322  9866 sys_catalog.cc:458] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:45.808172  9867 sys_catalog.cc:455] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a383cf8f549e4ebe990468b3cc2bb7c1. Latest consensus state: current_term: 1 leader_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a383cf8f549e4ebe990468b3cc2bb7c1" member_type: VOTER } }
I20260709 12:08:45.809069  9867 sys_catalog.cc:458] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:45.815105  9872 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:08:45.824623  9872 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:08:45.825713  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:08:45.834646  9872 catalog_manager.cc:1383] Generated new cluster ID: f877891914c94bfda6287b708c3f7e3f
I20260709 12:08:45.835041  9872 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:08:45.858477  9872 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:08:45.859946  9872 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:08:45.876014  9872 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1: Generated new TSK 0
I20260709 12:08:45.876703  9872 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:08:45.893091  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:45.900760  9883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:45.900990  9884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:45.904557  9261 server_base.cc:1061] running on GCE node
W20260709 12:08:45.905369  9887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:45.906503  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:45.906754  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:45.906989  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598925906947 us; error 0 us; skew 500 ppm
I20260709 12:08:45.907763  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:45.910522  9261 webserver.cc:533] Webserver started at http://127.9.11.65:33677/ using document root <none> and password file <none>
I20260709 12:08:45.911131  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:45.911357  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:45.911662  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:45.912974  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "a24b6211f48044f290dabea67c80438b"
format_stamp: "Formatted at 2026-07-09 12:08:45 on dist-test-slave-k289"
I20260709 12:08:45.917963  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:08:45.921811  9891 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:45.922672  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:08:45.923040  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "a24b6211f48044f290dabea67c80438b"
format_stamp: "Formatted at 2026-07-09 12:08:45 on dist-test-slave-k289"
I20260709 12:08:45.923327  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:45.935933  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:45.937417  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:45.939178  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:45.941736  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:45.941990  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:45.942256  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:45.942466  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:45.989342  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:43595
I20260709 12:08:45.989454  9953 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:43595 every 8 connection(s)
I20260709 12:08:45.994971  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:46.002600  9958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.004633  9959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.006366  9961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:46.007359  9261 server_base.cc:1061] running on GCE node
I20260709 12:08:46.009656  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:46.009974  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:46.009948  9954 heartbeater.cc:344] Connected to a master server at 127.9.11.126:44893
I20260709 12:08:46.010349  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598926010314 us; error 0 us; skew 500 ppm
I20260709 12:08:46.010569  9954 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:46.011076  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:46.011548  9954 heartbeater.cc:507] Master 127.9.11.126:44893 requested a full tablet report, sending...
I20260709 12:08:46.014125  9261 webserver.cc:533] Webserver started at http://127.9.11.66:35175/ using document root <none> and password file <none>
I20260709 12:08:46.014334  9827 ts_manager.cc:194] Registered new tserver with Master: a24b6211f48044f290dabea67c80438b (127.9.11.65:43595)
I20260709 12:08:46.014739  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:46.015048  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:46.015359  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:46.016590  9827 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47656
I20260709 12:08:46.016934  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "ed4d03d497fc443bb5c152dada8e137c"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-k289"
I20260709 12:08:46.021956  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.006s
I20260709 12:08:46.025653  9966 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.026499  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:08:46.026815  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "ed4d03d497fc443bb5c152dada8e137c"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-k289"
I20260709 12:08:46.027215  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:46.037039  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:46.038429  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:46.040099  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:46.042676  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:46.042938  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.043275  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:46.043486  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.089437  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:40509
I20260709 12:08:46.089581 10028 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:40509 every 8 connection(s)
I20260709 12:08:46.095045  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:46.102576 10032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.104219 10033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.106413 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:46.107651  9261 server_base.cc:1061] running on GCE node
I20260709 12:08:46.109223  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:08:46.109452 10029 heartbeater.cc:344] Connected to a master server at 127.9.11.126:44893
W20260709 12:08:46.109519  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:46.109920  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598926109894 us; error 0 us; skew 500 ppm
I20260709 12:08:46.109997 10029 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:46.110881  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:46.111231 10029 heartbeater.cc:507] Master 127.9.11.126:44893 requested a full tablet report, sending...
I20260709 12:08:46.114018  9261 webserver.cc:533] Webserver started at http://127.9.11.67:37471/ using document root <none> and password file <none>
I20260709 12:08:46.114145  9827 ts_manager.cc:194] Registered new tserver with Master: ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:46.114889  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:46.115245  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:46.115640  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:46.116495  9827 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47660
I20260709 12:08:46.117528  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "36fe98593e134396a04a0350038d7791"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-k289"
I20260709 12:08:46.123492  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:08:46.127792 10040 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.128692  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:08:46.129097  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "36fe98593e134396a04a0350038d7791"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-k289"
I20260709 12:08:46.129443  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:46.151399  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:46.152974  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:46.154716  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:46.157390  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:46.157647  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.157919  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:46.158113  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.205494  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:44403
I20260709 12:08:46.205629 10102 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:44403 every 8 connection(s)
I20260709 12:08:46.220887 10103 heartbeater.cc:344] Connected to a master server at 127.9.11.126:44893
I20260709 12:08:46.221339 10103 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:46.222199 10103 heartbeater.cc:507] Master 127.9.11.126:44893 requested a full tablet report, sending...
I20260709 12:08:46.224260  9827 ts_manager.cc:194] Registered new tserver with Master: 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:46.224848  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015833395s
I20260709 12:08:46.225893  9827 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47668
I20260709 12:08:46.254745  9827 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47682:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:08:46.258095  9827 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:08:46.360592 10067 tablet_service.cc:1511] Processing CreateTablet for tablet 84d8bfb3a4fc4f08a5be32940918d0c9 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:08:46.362365 10067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84d8bfb3a4fc4f08a5be32940918d0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.363212  9919 tablet_service.cc:1511] Processing CreateTablet for tablet fcd06b1b654243dabda6fb8210205d0e (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:08:46.364298 10066 tablet_service.cc:1511] Processing CreateTablet for tablet 988a4159073b4d23b82eb66372abc4ce (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:08:46.364931  9919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd06b1b654243dabda6fb8210205d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.365860 10066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988a4159073b4d23b82eb66372abc4ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.366364  9915 tablet_service.cc:1511] Processing CreateTablet for tablet 552faba7084e4f3797c7a2ec992f6870 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:08:46.366532  9990 tablet_service.cc:1511] Processing CreateTablet for tablet 552faba7084e4f3797c7a2ec992f6870 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:08:46.367836  9915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552faba7084e4f3797c7a2ec992f6870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.368062  9990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552faba7084e4f3797c7a2ec992f6870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.368304 10065 tablet_service.cc:1511] Processing CreateTablet for tablet 24436744f48143448904edb1894af2e1 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:08:46.369756 10065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24436744f48143448904edb1894af2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.369647  9917 tablet_service.cc:1511] Processing CreateTablet for tablet 988a4159073b4d23b82eb66372abc4ce (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:08:46.371187  9917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988a4159073b4d23b82eb66372abc4ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.364962  9916 tablet_service.cc:1511] Processing CreateTablet for tablet 24436744f48143448904edb1894af2e1 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:08:46.372653  9916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24436744f48143448904edb1894af2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.372416  9994 tablet_service.cc:1511] Processing CreateTablet for tablet fcd06b1b654243dabda6fb8210205d0e (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:08:46.373874  9994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd06b1b654243dabda6fb8210205d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.376371  9993 tablet_service.cc:1511] Processing CreateTablet for tablet 84d8bfb3a4fc4f08a5be32940918d0c9 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:08:46.377913  9993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84d8bfb3a4fc4f08a5be32940918d0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.378535  9989 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf2f2ddecfd43aaa11525e193f5f09b (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:08:46.380383  9992 tablet_service.cc:1511] Processing CreateTablet for tablet 988a4159073b4d23b82eb66372abc4ce (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:08:46.382607  9991 tablet_service.cc:1511] Processing CreateTablet for tablet 24436744f48143448904edb1894af2e1 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:08:46.359225 10068 tablet_service.cc:1511] Processing CreateTablet for tablet fcd06b1b654243dabda6fb8210205d0e (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:08:46.385289 10068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd06b1b654243dabda6fb8210205d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.385528  9914 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf2f2ddecfd43aaa11525e193f5f09b (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:08:46.387027  9914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf2f2ddecfd43aaa11525e193f5f09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.368307  9918 tablet_service.cc:1511] Processing CreateTablet for tablet 84d8bfb3a4fc4f08a5be32940918d0c9 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:08:46.388600  9918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84d8bfb3a4fc4f08a5be32940918d0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.394129 10064 tablet_service.cc:1511] Processing CreateTablet for tablet 552faba7084e4f3797c7a2ec992f6870 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:08:46.395603 10064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552faba7084e4f3797c7a2ec992f6870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.397727  9913 tablet_service.cc:1511] Processing CreateTablet for tablet db68b5b7900243b090255540436c6791 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:08:46.398618  9988 tablet_service.cc:1511] Processing CreateTablet for tablet db68b5b7900243b090255540436c6791 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:08:46.399259  9913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db68b5b7900243b090255540436c6791. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.400328 10063 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf2f2ddecfd43aaa11525e193f5f09b (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:08:46.401764 10063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf2f2ddecfd43aaa11525e193f5f09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.405894 10062 tablet_service.cc:1511] Processing CreateTablet for tablet db68b5b7900243b090255540436c6791 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:08:46.407692 10062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db68b5b7900243b090255540436c6791. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.408555  9987 tablet_service.cc:1511] Processing CreateTablet for tablet 76e37bffbed747f2b2280fbf995d7c0d (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:08:46.414518  9912 tablet_service.cc:1511] Processing CreateTablet for tablet 76e37bffbed747f2b2280fbf995d7c0d (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:08:46.416038  9912 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76e37bffbed747f2b2280fbf995d7c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.415617 10061 tablet_service.cc:1511] Processing CreateTablet for tablet 76e37bffbed747f2b2280fbf995d7c0d (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:08:46.417130 10061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76e37bffbed747f2b2280fbf995d7c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.426658  9989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf2f2ddecfd43aaa11525e193f5f09b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.427786  9992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 988a4159073b4d23b82eb66372abc4ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.429350  9991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24436744f48143448904edb1894af2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.432068  9988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db68b5b7900243b090255540436c6791. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.433457  9987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76e37bffbed747f2b2280fbf995d7c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.436013 10118 tablet_bootstrap.cc:492] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.454447  9913 tablet_service.cc:1511] Processing CreateTablet for tablet 41aa7a0f0fb74559be3977acd9d10dce (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:08:46.456691  9913 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41aa7a0f0fb74559be3977acd9d10dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.492009  9990 tablet_service.cc:1511] Processing CreateTablet for tablet 41aa7a0f0fb74559be3977acd9d10dce (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:08:46.493646  9990 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41aa7a0f0fb74559be3977acd9d10dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.501897 10119 tablet_bootstrap.cc:492] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:46.505271 10060 tablet_service.cc:1511] Processing CreateTablet for tablet 41aa7a0f0fb74559be3977acd9d10dce (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:08:46.506974 10060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41aa7a0f0fb74559be3977acd9d10dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.511535  9917 tablet_service.cc:1511] Processing CreateTablet for tablet d025b065de3149af9dbe4e8665934bd1 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:08:46.528013 10118 tablet_bootstrap.cc:654] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.528620  9917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d025b065de3149af9dbe4e8665934bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.580559  9986 tablet_service.cc:1511] Processing CreateTablet for tablet d025b065de3149af9dbe4e8665934bd1 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:08:46.582370  9986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d025b065de3149af9dbe4e8665934bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.588546 10059 tablet_service.cc:1511] Processing CreateTablet for tablet d025b065de3149af9dbe4e8665934bd1 (DEFAULT_TABLE table=test-workload [id=eda69080b95d4b088899cfb1daf7038c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:08:46.590132 10059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d025b065de3149af9dbe4e8665934bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.621440 10118 tablet_bootstrap.cc:492] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:46.622013 10118 ts_tablet_manager.cc:1403] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.000s
I20260709 12:08:46.624174 10119 tablet_bootstrap.cc:654] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.625161 10118 raft_consensus.cc:359] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.625958 10118 raft_consensus.cc:385] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.626288 10118 raft_consensus.cc:740] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.650739 10120 tablet_bootstrap.cc:492] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:46.657898 10118 consensus_queue.cc:260] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.663959 10119 tablet_bootstrap.cc:492] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:46.664588 10119 ts_tablet_manager.cc:1403] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.163s	user 0.005s	sys 0.014s
I20260709 12:08:46.664635 10120 tablet_bootstrap.cc:654] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.668201 10118 ts_tablet_manager.cc:1434] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.000s
I20260709 12:08:46.669121 10029 heartbeater.cc:499] Master 127.9.11.126:44893 was elected leader, sending a full tablet report...
I20260709 12:08:46.669377 10118 tablet_bootstrap.cc:492] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.671978 10119 raft_consensus.cc:359] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.672859 10119 raft_consensus.cc:385] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.673193 10119 raft_consensus.cc:740] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.674048 10119 consensus_queue.cc:260] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.678200 10118 tablet_bootstrap.cc:654] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.682454 10120 tablet_bootstrap.cc:492] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:46.683130 10120 ts_tablet_manager.cc:1403] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.006s
I20260709 12:08:46.684087 10119 ts_tablet_manager.cc:1434] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.005s
I20260709 12:08:46.684204  9954 heartbeater.cc:499] Master 127.9.11.126:44893 was elected leader, sending a full tablet report...
I20260709 12:08:46.685253 10119 tablet_bootstrap.cc:492] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:46.686859 10120 raft_consensus.cc:359] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.687788 10120 raft_consensus.cc:385] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.688212 10120 raft_consensus.cc:740] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.689424 10120 consensus_queue.cc:260] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.693436 10119 tablet_bootstrap.cc:654] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.701740 10118 tablet_bootstrap.cc:492] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:46.702275 10118 ts_tablet_manager.cc:1403] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.010s
I20260709 12:08:46.706480 10103 heartbeater.cc:499] Master 127.9.11.126:44893 was elected leader, sending a full tablet report...
I20260709 12:08:46.706425 10118 raft_consensus.cc:359] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.707592 10118 raft_consensus.cc:385] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.707991 10118 raft_consensus.cc:740] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.708916 10118 consensus_queue.cc:260] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.710291 10120 ts_tablet_manager.cc:1434] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.012s
I20260709 12:08:46.711511 10120 tablet_bootstrap.cc:492] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:46.711838 10118 ts_tablet_manager.cc:1434] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.005s
I20260709 12:08:46.712931 10118 tablet_bootstrap.cc:492] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.716015 10119 tablet_bootstrap.cc:492] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:46.716775 10119 ts_tablet_manager.cc:1403] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.006s
I20260709 12:08:46.720593 10119 raft_consensus.cc:359] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:46.722048 10119 raft_consensus.cc:385] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.722414 10119 raft_consensus.cc:740] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.723384 10119 consensus_queue.cc:260] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:46.726040 10119 ts_tablet_manager.cc:1434] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 12:08:46.726377 10125 raft_consensus.cc:493] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:46.727212 10119 tablet_bootstrap.cc:492] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:46.726913 10125 raft_consensus.cc:515] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:46.728266 10120 tablet_bootstrap.cc:654] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.721680 10118 tablet_bootstrap.cc:654] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.745235 10125 leader_election.cc:290] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:46.750277 10119 tablet_bootstrap.cc:654] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.765400 10120 tablet_bootstrap.cc:492] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:46.766057 10120 ts_tablet_manager.cc:1403] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.055s	user 0.023s	sys 0.016s
I20260709 12:08:46.766255 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:46.767387 10118 tablet_bootstrap.cc:492] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:46.767982 10118 ts_tablet_manager.cc:1403] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.015s
I20260709 12:08:46.769114 10120 raft_consensus.cc:359] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.769703 10120 raft_consensus.cc:385] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.770051 10120 raft_consensus.cc:740] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.771219 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:46.771621 10118 raft_consensus.cc:359] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.771082 10120 consensus_queue.cc:260] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.777650 10118 raft_consensus.cc:385] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.778034 10118 raft_consensus.cc:740] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
W20260709 12:08:46.770529  9893 leader_election.cc:343] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:46.779119 10118 consensus_queue.cc:260] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
W20260709 12:08:46.779719  9894 leader_election.cc:343] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:46.780212  9894 leader_election.cc:304] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a24b6211f48044f290dabea67c80438b; no voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c
I20260709 12:08:46.781636 10125 raft_consensus.cc:2749] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:46.781800 10118 ts_tablet_manager.cc:1434] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 12:08:46.784672 10120 ts_tablet_manager.cc:1434] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.005s
I20260709 12:08:46.785768 10120 tablet_bootstrap.cc:492] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:46.786429 10119 tablet_bootstrap.cc:492] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:46.786957 10119 ts_tablet_manager.cc:1403] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.010s
I20260709 12:08:46.788335 10118 tablet_bootstrap.cc:492] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.790488 10125 raft_consensus.cc:493] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:46.790582 10119 raft_consensus.cc:359] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.791551 10119 raft_consensus.cc:385] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.791919 10119 raft_consensus.cc:740] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.792068 10125 raft_consensus.cc:515] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.792948 10119 consensus_queue.cc:260] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.796700 10118 tablet_bootstrap.cc:654] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.796803 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:46.798219 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:46.798960 10004 raft_consensus.cc:2468] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 0.
W20260709 12:08:46.799659  9894 leader_election.cc:343] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:46.805253 10125 leader_election.cc:290] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:46.805505 10119 ts_tablet_manager.cc:1434] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.012s
I20260709 12:08:46.806427  9893 leader_election.cc:304] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 36fe98593e134396a04a0350038d7791
I20260709 12:08:46.808059 10118 tablet_bootstrap.cc:492] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:46.808485 10119 tablet_bootstrap.cc:492] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:46.808630 10118 ts_tablet_manager.cc:1403] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.009s
I20260709 12:08:46.812738 10120 tablet_bootstrap.cc:654] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.812361 10118 raft_consensus.cc:359] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.813334 10118 raft_consensus.cc:385] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.813805 10118 raft_consensus.cc:740] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.814913 10118 consensus_queue.cc:260] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
W20260709 12:08:46.816563  9875 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:46.817698 10118 ts_tablet_manager.cc:1434] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:08:46.818902 10118 tablet_bootstrap.cc:492] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.820914 10119 tablet_bootstrap.cc:654] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.821413 10134 raft_consensus.cc:2804] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:46.821818 10134 raft_consensus.cc:493] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:46.822202 10134 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:46.826802 10118 tablet_bootstrap.cc:654] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.834268 10134 raft_consensus.cc:515] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.835661 10120 tablet_bootstrap.cc:492] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:46.836319 10120 ts_tablet_manager.cc:1403] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.004s
I20260709 12:08:46.837563 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:46.838227 10004 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:46.839977 10120 raft_consensus.cc:359] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:46.840795 10120 raft_consensus.cc:385] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.841157 10120 raft_consensus.cc:740] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.843449 10120 consensus_queue.cc:260] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:46.845922 10120 ts_tablet_manager.cc:1434] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:08:46.846913 10118 tablet_bootstrap.cc:492] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:46.847683 10118 ts_tablet_manager.cc:1403] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.016s
I20260709 12:08:46.849084 10120 tablet_bootstrap.cc:492] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:46.851990 10118 raft_consensus.cc:359] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.852813 10118 raft_consensus.cc:385] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.853179 10118 raft_consensus.cc:740] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.854246 10004 raft_consensus.cc:2468] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 1.
I20260709 12:08:46.854995 10127 raft_consensus.cc:493] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:46.854188 10118 consensus_queue.cc:260] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.854236 10119 tablet_bootstrap.cc:492] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:46.855496 10127 raft_consensus.cc:515] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.856149 10119 ts_tablet_manager.cc:1403] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.018s
I20260709 12:08:46.859881 10119 raft_consensus.cc:359] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.860937 10119 raft_consensus.cc:385] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.861208 10118 ts_tablet_manager.cc:1434] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.005s
I20260709 12:08:46.861330 10120 tablet_bootstrap.cc:654] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.861328 10119 raft_consensus.cc:740] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.862499 10118 tablet_bootstrap.cc:492] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.862535 10119 consensus_queue.cc:260] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.867899 10119 ts_tablet_manager.cc:1434] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:08:46.869167 10119 tablet_bootstrap.cc:492] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:46.871184 10118 tablet_bootstrap.cc:654] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.871445  9893 leader_election.cc:304] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:46.873169 10127 leader_election.cc:290] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:46.873953 10125 raft_consensus.cc:2804] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:46.892736 10119 tablet_bootstrap.cc:654] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.896690 10120 tablet_bootstrap.cc:492] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:46.897313 10120 ts_tablet_manager.cc:1403] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.012s
I20260709 12:08:46.889127 10134 leader_election.cc:290] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:46.906665 10120 raft_consensus.cc:359] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.903354 10125 raft_consensus.cc:697] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:46.909361 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:46.907809 10120 raft_consensus.cc:385] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.910153 10120 raft_consensus.cc:740] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.913703 10120 consensus_queue.cc:260] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.915707  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:46.916600  9929 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
W20260709 12:08:46.916963  9894 leader_election.cc:343] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:46.918406 10044 leader_election.cc:304] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:08:46.919440 10127 raft_consensus.cc:2804] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:46.920032 10127 raft_consensus.cc:493] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:46.920496 10127 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:46.920590 10125 consensus_queue.cc:237] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.925212 10118 tablet_bootstrap.cc:492] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:46.925814 10118 ts_tablet_manager.cc:1403] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.021s
I20260709 12:08:46.929214 10118 raft_consensus.cc:359] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.929966 10118 raft_consensus.cc:385] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.930079 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:46.930338 10118 raft_consensus.cc:740] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.931317 10118 consensus_queue.cc:260] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
W20260709 12:08:46.931810 10042 leader_election.cc:343] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:46.929456 10127 raft_consensus.cc:515] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.935600  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:46.935976 10123 raft_consensus.cc:493] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:46.936233  9929 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:46.936484 10123 raft_consensus.cc:515] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.944105  9929 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:46.945580 10044 leader_election.cc:304] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:08:46.953528 10123 leader_election.cc:290] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:46.953840 10119 tablet_bootstrap.cc:492] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:46.954425 10119 ts_tablet_manager.cc:1403] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.085s	user 0.016s	sys 0.006s
I20260709 12:08:46.959870 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
W20260709 12:08:46.962026 10042 leader_election.cc:343] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:46.966254 10118 ts_tablet_manager.cc:1434] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.002s
I20260709 12:08:46.968008 10119 raft_consensus.cc:359] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.969043 10119 raft_consensus.cc:385] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.969412 10119 raft_consensus.cc:740] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.970304 10127 leader_election.cc:290] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:46.971385 10127 raft_consensus.cc:2804] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:46.971881 10127 raft_consensus.cc:697] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:46.972764 10127 consensus_queue.cc:237] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:46.973814 10118 tablet_bootstrap.cc:492] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:46.980832 10119 consensus_queue.cc:260] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.002300 10120 ts_tablet_manager.cc:1434] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.105s	user 0.037s	sys 0.042s
I20260709 12:08:47.003973 10120 tablet_bootstrap.cc:492] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:47.007093 10119 ts_tablet_manager.cc:1434] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.004s
I20260709 12:08:47.007447 10123 raft_consensus.cc:493] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.008250 10119 tablet_bootstrap.cc:492] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:47.007941 10123 raft_consensus.cc:515] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.008817  9827 catalog_manager.cc:5697] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 0 to 1, leader changed from <none> to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 1 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.012542 10123 leader_election.cc:290] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.016009  9824 catalog_manager.cc:5697] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:47.017558 10119 tablet_bootstrap.cc:654] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.017642 10120 tablet_bootstrap.cc:654] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.031754 10118 tablet_bootstrap.cc:654] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.032299 10123 raft_consensus.cc:493] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.032727 10119 tablet_bootstrap.cc:492] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:47.032848 10123 raft_consensus.cc:515] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.034571 10120 tablet_bootstrap.cc:492] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:47.035122 10120 ts_tablet_manager.cc:1403] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.013s
I20260709 12:08:47.035259 10123 leader_election.cc:290] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.038507 10120 raft_consensus.cc:359] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.039326 10120 raft_consensus.cc:385] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.039646 10120 raft_consensus.cc:740] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.039752 10118 tablet_bootstrap.cc:492] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:47.040364 10118 ts_tablet_manager.cc:1403] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.067s	user 0.008s	sys 0.007s
I20260709 12:08:47.040606 10120 consensus_queue.cc:260] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.043540 10120 ts_tablet_manager.cc:1434] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:08:47.043756 10118 raft_consensus.cc:359] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.044514 10120 tablet_bootstrap.cc:492] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:47.044605 10118 raft_consensus.cc:385] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.045028 10118 raft_consensus.cc:740] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.045842 10118 consensus_queue.cc:260] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.049461 10118 ts_tablet_manager.cc:1434] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:08:47.050812 10118 tablet_bootstrap.cc:492] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:47.051745 10120 tablet_bootstrap.cc:654] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.033283 10119 ts_tablet_manager.cc:1403] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.002s
I20260709 12:08:47.060117 10118 tablet_bootstrap.cc:654] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.061069 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.064139 10123 raft_consensus.cc:493] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.064814 10123 raft_consensus.cc:515] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.067545 10120 tablet_bootstrap.cc:492] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:47.067847 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.068075 10120 ts_tablet_manager.cc:1403] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:08:47.068542 10078 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.066927 10123 leader_election.cc:290] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.071118 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.071471 10120 raft_consensus.cc:359] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.072135 10118 tablet_bootstrap.cc:492] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:47.072233 10120 raft_consensus.cc:385] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.072719 10120 raft_consensus.cc:740] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.072877 10118 ts_tablet_manager.cc:1403] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 12:08:47.073813 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.073566 10120 consensus_queue.cc:260] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.074566 10078 raft_consensus.cc:2468] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.076359 10120 ts_tablet_manager.cc:1434] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:08:47.077423 10120 tablet_bootstrap.cc:492] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:47.076934 10118 raft_consensus.cc:359] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.077850 10118 raft_consensus.cc:385] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.078199 10118 raft_consensus.cc:740] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.079322 10118 consensus_queue.cc:260] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
W20260709 12:08:47.080669  9969 leader_election.cc:343] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.081612  9969 leader_election.cc:304] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.083899 10119 raft_consensus.cc:359] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.084858 10120 tablet_bootstrap.cc:654] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.084971 10119 raft_consensus.cc:385] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.085461 10119 raft_consensus.cc:740] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.086537 10119 consensus_queue.cc:260] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.089769 10119 ts_tablet_manager.cc:1434] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:08:47.082458 10118 ts_tablet_manager.cc:1434] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 12:08:47.091189  9926 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.091866 10118 tablet_bootstrap.cc:492] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:47.091643  9928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.091902  9926 raft_consensus.cc:2468] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.092590  9928 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.093510  9970 leader_election.cc:304] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.094238  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.095757  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.099264 10123 raft_consensus.cc:2804] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:08:47.099413  9969 leader_election.cc:343] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.099717 10123 raft_consensus.cc:493] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.100160 10123 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.102049 10120 tablet_bootstrap.cc:492] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:47.102708 10120 ts_tablet_manager.cc:1403] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.009s
I20260709 12:08:47.106415 10120 raft_consensus.cc:359] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.107295 10120 raft_consensus.cc:385] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.107666 10120 raft_consensus.cc:740] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.108702 10120 consensus_queue.cc:260] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.110922 10118 tablet_bootstrap.cc:654] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.111521 10120 ts_tablet_manager.cc:1434] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:08:47.112668 10120 tablet_bootstrap.cc:492] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:47.116819 10159 raft_consensus.cc:2804] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.117412 10159 raft_consensus.cc:493] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.117894 10159 raft_consensus.cc:3060] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:08:47.117780  9970 leader_election.cc:343] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a24b6211f48044f290dabea67c80438b (127.9.11.65:43595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.118671  9970 leader_election.cc:304] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed4d03d497fc443bb5c152dada8e137c; no voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.121598 10119 tablet_bootstrap.cc:492] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:47.121259 10123 raft_consensus.cc:515] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.123456 10120 tablet_bootstrap.cc:654] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:47.127553  9970 leader_election.cc:343] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a24b6211f48044f290dabea67c80438b (127.9.11.65:43595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.128147  9970 leader_election.cc:304] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed4d03d497fc443bb5c152dada8e137c; no voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.129132 10123 leader_election.cc:290] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.129694 10123 raft_consensus.cc:2749] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.129945 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.130573 10123 raft_consensus.cc:493] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.130648 10078 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.131435 10118 tablet_bootstrap.cc:492] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:47.134999 10123 raft_consensus.cc:515] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.134675 10161 raft_consensus.cc:2749] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.135439 10118 ts_tablet_manager.cc:1403] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.014s
I20260709 12:08:47.140188 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d025b065de3149af9dbe4e8665934bd1" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
W20260709 12:08:47.142625  9969 leader_election.cc:343] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.145967 10119 tablet_bootstrap.cc:654] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.147837 10159 raft_consensus.cc:515] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.148677 10078 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.141903 10161 raft_consensus.cc:493] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.149752 10118 raft_consensus.cc:359] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.150579 10118 raft_consensus.cc:385] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.150926 10118 raft_consensus.cc:740] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.150450 10161 raft_consensus.cc:515] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.151733 10118 consensus_queue.cc:260] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.153719 10161 leader_election.cc:290] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.153959 10118 ts_tablet_manager.cc:1434] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 12:08:47.154516  9969 leader_election.cc:304] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.156674 10118 tablet_bootstrap.cc:492] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: Bootstrap starting.
I20260709 12:08:47.139422 10123 leader_election.cc:290] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.157241 10161 raft_consensus.cc:2804] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.158766 10161 raft_consensus.cc:697] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.178551 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.179545 10077 raft_consensus.cc:3060] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.178996 10161 consensus_queue.cc:237] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.159185 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.189024 10078 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.186661 10159 leader_election.cc:290] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
W20260709 12:08:47.187593  9970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:43595, user_credentials={real_user=slave}} blocked reactor thread for 59445.1us
I20260709 12:08:47.192082  9969 leader_election.cc:304] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.194083 10159 raft_consensus.cc:2804] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.194533 10159 raft_consensus.cc:493] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.194900 10159 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.194828  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.195626  9929 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.195884 10119 tablet_bootstrap.cc:492] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:47.196758 10119 ts_tablet_manager.cc:1403] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.016s
I20260709 12:08:47.201067 10119 raft_consensus.cc:359] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.201781 10077 raft_consensus.cc:2468] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.203451  9969 leader_election.cc:304] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.204493 10120 tablet_bootstrap.cc:492] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:47.204977 10120 ts_tablet_manager.cc:1403] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.093s	user 0.025s	sys 0.012s
I20260709 12:08:47.206002  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d025b065de3149af9dbe4e8665934bd1" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.202245 10119 raft_consensus.cc:385] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.207497  9928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.208148  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.208213  9928 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.208259 10120 raft_consensus.cc:359] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.208841  9927 raft_consensus.cc:3060] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.209052 10120 raft_consensus.cc:385] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.209394 10120 raft_consensus.cc:740] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.206167  9929 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.206861 10163 raft_consensus.cc:2804] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.210229 10120 consensus_queue.cc:260] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.211035 10163 raft_consensus.cc:697] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.211978 10163 consensus_queue.cc:237] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.211715  9827 catalog_manager.cc:5697] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 1 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.212707 10120 ts_tablet_manager.cc:1434] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:08:47.209618 10119 raft_consensus.cc:740] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.217149 10120 tablet_bootstrap.cc:492] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:47.218158 10119 consensus_queue.cc:260] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.222440  9927 raft_consensus.cc:2468] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.226274 10119 ts_tablet_manager.cc:1434] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 12:08:47.227617 10120 tablet_bootstrap.cc:654] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.228194 10119 tablet_bootstrap.cc:492] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
W20260709 12:08:47.226927  9970 leader_election.cc:343] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a24b6211f48044f290dabea67c80438b (127.9.11.65:43595): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.228791  9970 leader_election.cc:304] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed4d03d497fc443bb5c152dada8e137c; no voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.231549 10161 raft_consensus.cc:2749] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.238015 10119 tablet_bootstrap.cc:654] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.238703 10118 tablet_bootstrap.cc:654] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.248800  9824 catalog_manager.cc:5697] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 1 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.255306 10119 tablet_bootstrap.cc:492] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:47.255887 10119 ts_tablet_manager.cc:1403] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.012s
I20260709 12:08:47.242694 10159 raft_consensus.cc:515] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.259193 10119 raft_consensus.cc:359] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.259945 10119 raft_consensus.cc:385] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.260246 10119 raft_consensus.cc:740] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.260780  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.261538 10119 consensus_queue.cc:260] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.261421  9927 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.262506 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.263137 10077 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.261605 10159 leader_election.cc:290] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.264616 10119 ts_tablet_manager.cc:1434] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:08:47.265640 10119 tablet_bootstrap.cc:492] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:47.271219 10077 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.272362 10119 tablet_bootstrap.cc:654] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.272944  9969 leader_election.cc:304] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.273936 10159 raft_consensus.cc:2804] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.274469 10159 raft_consensus.cc:697] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.275547 10118 tablet_bootstrap.cc:492] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: No bootstrap required, opened a new log
I20260709 12:08:47.275420 10159 consensus_queue.cc:237] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.276201  9927 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.275565 10120 tablet_bootstrap.cc:492] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:47.277432 10120 ts_tablet_manager.cc:1403] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.060s	user 0.026s	sys 0.006s
I20260709 12:08:47.279994 10120 raft_consensus.cc:359] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.280545 10120 raft_consensus.cc:385] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.280818 10120 raft_consensus.cc:740] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.276064 10118 ts_tablet_manager.cc:1403] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.001s
I20260709 12:08:47.281425 10120 consensus_queue.cc:260] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.283802 10120 ts_tablet_manager.cc:1434] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260709 12:08:47.284895 10120 tablet_bootstrap.cc:492] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791: Bootstrap starting.
I20260709 12:08:47.284790 10118 raft_consensus.cc:359] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.285601 10118 raft_consensus.cc:385] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.285918 10118 raft_consensus.cc:740] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.286616 10118 consensus_queue.cc:260] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.288628 10145 raft_consensus.cc:493] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.289261 10118 ts_tablet_manager.cc:1434] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:08:47.289096  9824 catalog_manager.cc:5697] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 1 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.292142 10120 tablet_bootstrap.cc:654] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.289127 10145 raft_consensus.cc:515] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.293206 10119 tablet_bootstrap.cc:492] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:47.294725 10119 ts_tablet_manager.cc:1403] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.002s
I20260709 12:08:47.295770 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d025b065de3149af9dbe4e8665934bd1" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:47.296427 10004 raft_consensus.cc:2468] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
I20260709 12:08:47.297808 10119 raft_consensus.cc:359] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.298487 10119 raft_consensus.cc:385] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.298769 10119 raft_consensus.cc:740] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.299042 10042 leader_election.cc:304] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.299981 10127 raft_consensus.cc:2804] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.299793 10119 consensus_queue.cc:260] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.300354 10127 raft_consensus.cc:493] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.300742 10127 raft_consensus.cc:3060] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.302287 10119 ts_tablet_manager.cc:1434] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:08:47.303288 10119 tablet_bootstrap.cc:492] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b: Bootstrap starting.
I20260709 12:08:47.306334  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d025b065de3149af9dbe4e8665934bd1" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.307062  9927 raft_consensus.cc:2468] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
I20260709 12:08:47.310386 10120 tablet_bootstrap.cc:492] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791: No bootstrap required, opened a new log
I20260709 12:08:47.310567 10119 tablet_bootstrap.cc:654] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.310899 10120 ts_tablet_manager.cc:1403] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.001s
I20260709 12:08:47.311388 10148 raft_consensus.cc:493] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.311928 10148 raft_consensus.cc:515] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.314272 10148 leader_election.cc:290] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.313975 10120 raft_consensus.cc:359] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.314699 10120 raft_consensus.cc:385] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.315030 10120 raft_consensus.cc:740] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.315780 10120 consensus_queue.cc:260] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.316193  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.316635 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:47.317098  9927 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
I20260709 12:08:47.317236 10004 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
I20260709 12:08:47.317947 10120 ts_tablet_manager.cc:1434] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:08:47.318773 10042 leader_election.cc:304] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.320870 10176 raft_consensus.cc:2804] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.321377 10176 raft_consensus.cc:493] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.321789 10176 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.323406 10145 leader_election.cc:290] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.323589 10119 tablet_bootstrap.cc:492] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b: No bootstrap required, opened a new log
I20260709 12:08:47.324168 10119 ts_tablet_manager.cc:1403] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 12:08:47.327452 10119 raft_consensus.cc:359] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.328194 10119 raft_consensus.cc:385] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.328469 10119 raft_consensus.cc:740] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.329140 10119 consensus_queue.cc:260] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.329581 10176 raft_consensus.cc:515] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.329771 10127 raft_consensus.cc:515] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.331422 10119 ts_tablet_manager.cc:1434] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:08:47.331508 10125 consensus_queue.cc:1048] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.332892  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.333011 10176 leader_election.cc:290] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.333536  9927 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.334695 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.335299 10004 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.341436  9927 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:47.342648 10044 leader_election.cc:304] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:08:47.343663 10176 raft_consensus.cc:2804] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.344215 10176 raft_consensus.cc:697] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:47.345155 10176 consensus_queue.cc:237] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
W20260709 12:08:47.351843 10030 tablet.cc:2406] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:08:47.353020 10004 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:47.357120 10127 leader_election.cc:290] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.357941  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d025b065de3149af9dbe4e8665934bd1" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.358657  9927 raft_consensus.cc:3060] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.360023 10125 raft_consensus.cc:493] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.360702 10125 raft_consensus.cc:515] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.364957 10159 raft_consensus.cc:493] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.365532 10159 raft_consensus.cc:515] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.364881 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:47.366602 10003 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 0.
I20260709 12:08:47.367219  9927 raft_consensus.cc:2468] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:47.367818 10159 leader_election.cc:290] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.368707 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d025b065de3149af9dbe4e8665934bd1" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.368482  9824 catalog_manager.cc:5697] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.369241 10044 leader_election.cc:304] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:08:47.369406 10004 raft_consensus.cc:3060] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.370222 10127 raft_consensus.cc:2804] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.370754 10127 raft_consensus.cc:697] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:47.371351 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.372056 10078 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.371376 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.373486 10077 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 0.
I20260709 12:08:47.373548  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.374276  9927 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 0.
I20260709 12:08:47.374712  9894 leader_election.cc:304] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:08:47.375659 10134 raft_consensus.cc:2804] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.375555  9970 leader_election.cc:304] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.376093 10134 raft_consensus.cc:493] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.376260 10077 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.376505 10134 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.377031 10159 raft_consensus.cc:2804] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.371681 10127 consensus_queue.cc:237] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.377444 10159 raft_consensus.cc:493] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.377861 10159 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.367857 10125 leader_election.cc:290] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.384481 10004 raft_consensus.cc:2468] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:47.385778 10159 raft_consensus.cc:515] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.388854  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.390465 10159 leader_election.cc:290] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.391202 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.391829 10078 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.393282 10125 consensus_queue.cc:1048] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.398254 10134 raft_consensus.cc:515] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.401912 10078 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 1.
I20260709 12:08:47.402419 10145 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.403196  9969 leader_election.cc:304] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.404193 10163 raft_consensus.cc:2804] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.404718 10163 raft_consensus.cc:697] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.405604 10163 consensus_queue.cc:237] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.407512  9927 raft_consensus.cc:2393] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ed4d03d497fc443bb5c152dada8e137c in current term 1: Already voted for candidate a24b6211f48044f290dabea67c80438b in this term.
I20260709 12:08:47.407416 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.408438 10134 leader_election.cc:290] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.410256 10076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.415241 10076 raft_consensus.cc:2393] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a24b6211f48044f290dabea67c80438b in current term 1: Already voted for candidate ed4d03d497fc443bb5c152dada8e137c in this term.
I20260709 12:08:47.417270 10125 raft_consensus.cc:493] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.417029  9894 leader_election.cc:304] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a24b6211f48044f290dabea67c80438b; no voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c
I20260709 12:08:47.417778 10125 raft_consensus.cc:515] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.419162 10134 raft_consensus.cc:2749] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.418871  9827 catalog_manager.cc:5697] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:47.420790 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:47.421111 10125 leader_election.cc:290] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.421625 10003 raft_consensus.cc:2468] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 0.
I20260709 12:08:47.423125  9893 leader_election.cc:304] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.425267 10076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791" is_pre_election: true
I20260709 12:08:47.426066 10076 raft_consensus.cc:2468] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 0.
I20260709 12:08:47.428628 10142 raft_consensus.cc:2804] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.429242 10142 raft_consensus.cc:493] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.429662 10142 raft_consensus.cc:3060] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.436687  9827 catalog_manager.cc:5697] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 0 to 1, leader changed from <none> to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 1 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.440446 10142 raft_consensus.cc:515] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.443797 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.444403 10003 raft_consensus.cc:3060] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.453444 10003 raft_consensus.cc:2468] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 1.
I20260709 12:08:47.455076  9893 leader_election.cc:304] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.456712 10125 raft_consensus.cc:2804] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.457206 10125 raft_consensus.cc:697] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:47.457767 10142 leader_election.cc:290] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 1 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.458047 10125 consensus_queue.cc:237] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.459206 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.460024 10078 raft_consensus.cc:3060] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.470336 10078 raft_consensus.cc:2468] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 1.
I20260709 12:08:47.472059 10145 raft_consensus.cc:493] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.472775 10145 raft_consensus.cc:515] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.476775 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:47.476704  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.477613 10003 raft_consensus.cc:2468] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
I20260709 12:08:47.477766  9929 raft_consensus.cc:2468] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 0.
I20260709 12:08:47.478298 10127 raft_consensus.cc:493] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a24b6211f48044f290dabea67c80438b)
I20260709 12:08:47.478997 10042 leader_election.cc:304] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.478849 10127 raft_consensus.cc:515] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.479933 10176 raft_consensus.cc:2804] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.480335 10176 raft_consensus.cc:493] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.480687 10176 raft_consensus.cc:3060] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.484536 10127 leader_election.cc:290] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.486747 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c" is_pre_election: true
I20260709 12:08:47.487363  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b" is_pre_election: true
I20260709 12:08:47.489674 10003 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.489533 10044 leader_election.cc:304] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36fe98593e134396a04a0350038d7791; no voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c
I20260709 12:08:47.491475 10127 raft_consensus.cc:2749] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:08:47.493418 10145 leader_election.cc:290] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.495929  9824 catalog_manager.cc:5697] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 0 to 1, leader changed from <none> to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 1 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.498860 10176 raft_consensus.cc:515] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.502086 10189 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.502908  9929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.503541  9929 raft_consensus.cc:3060] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.504200 10176 leader_election.cc:290] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.506366 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.507210 10004 raft_consensus.cc:3060] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.522627 10004 raft_consensus.cc:2468] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:47.523918 10042 leader_election.cc:304] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.524835 10176 raft_consensus.cc:2804] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.525520 10176 raft_consensus.cc:697] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:47.526432 10176 consensus_queue.cc:237] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.524712  9929 raft_consensus.cc:2468] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 1.
I20260709 12:08:47.537004  9827 catalog_manager.cc:5697] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:47.585754 10077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e"
dest_uuid: "36fe98593e134396a04a0350038d7791"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:38738
I20260709 12:08:47.586438 10077 raft_consensus.cc:606] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.591688 10077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9"
dest_uuid: "36fe98593e134396a04a0350038d7791"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:38738
I20260709 12:08:47.592224 10077 raft_consensus.cc:606] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.608652 10004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24436744f48143448904edb1894af2e1"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:58632
I20260709 12:08:47.609459 10004 raft_consensus.cc:606] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.615068 10004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:58632
I20260709 12:08:47.615706 10004 raft_consensus.cc:606] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:08:47.623250 10077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db68b5b7900243b090255540436c6791"
dest_uuid: "36fe98593e134396a04a0350038d7791"
mode: GRACEFUL
new_leader_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:38738
I20260709 12:08:47.623795 10077 raft_consensus.cc:606] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Received request to transfer leadership to TS ed4d03d497fc443bb5c152dada8e137c
I20260709 12:08:47.639565  9929 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:41390
I20260709 12:08:47.640125  9929 raft_consensus.cc:606] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Received request to transfer leadership to TS ed4d03d497fc443bb5c152dada8e137c
I20260709 12:08:47.652956 10163 consensus_queue.cc:1048] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.665237 10163 consensus_queue.cc:1048] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.685751 10161 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.685915 10163 consensus_queue.cc:1048] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.697073 10161 raft_consensus.cc:993] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:47.697584 10161 raft_consensus.cc:1081] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:47.701270  9927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41384
I20260709 12:08:47.701972  9927 raft_consensus.cc:493] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:47.702411  9927 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.704763 10163 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.706532 10163 consensus_queue.cc:1048] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.711198  9927 raft_consensus.cc:515] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.712738  9929 raft_consensus.cc:1240] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Rejecting Update request from peer ed4d03d497fc443bb5c152dada8e137c for earlier term 1. Current term is 2. Ops: []
I20260709 12:08:47.715104 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.715195 10163 consensus_queue.cc:1059] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.716121 10161 raft_consensus.cc:3055] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:47.716485 10161 raft_consensus.cc:740] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.717391 10161 consensus_queue.cc:260] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.718128 10161 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.717962 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.724586  9927 leader_election.cc:290] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.731702 10163 raft_consensus.cc:993] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:47.732228 10163 raft_consensus.cc:1081] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:47.734056  9928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24436744f48143448904edb1894af2e1"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41384
I20260709 12:08:47.734701  9928 raft_consensus.cc:493] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:47.734937 10004 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:08:47.735106  9928 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.738054  9893 leader_election.cc:304] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 36fe98593e134396a04a0350038d7791
I20260709 12:08:47.739127 10125 raft_consensus.cc:2804] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:47.739696 10125 raft_consensus.cc:697] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:47.740401 10125 consensus_queue.cc:237] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.741320  9928 raft_consensus.cc:515] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.743505  9928 leader_election.cc:290] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.744804 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.745452 10077 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.748347 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.749091 10004 raft_consensus.cc:3055] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:47.749788 10176 consensus_queue.cc:1048] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.749488 10004 raft_consensus.cc:740] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.751940 10004 consensus_queue.cc:260] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.752693 10077 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:08:47.753556 10004 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.754271  9894 leader_election.cc:304] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:08:47.756286 10125 raft_consensus.cc:2804] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:47.756989 10125 raft_consensus.cc:697] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:47.757893 10125 consensus_queue.cc:237] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.758136  9827 catalog_manager.cc:5697] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 1 to 2, leader changed from ed4d03d497fc443bb5c152dada8e137c (127.9.11.66) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 2 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.759529 10189 raft_consensus.cc:993] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:47.760030 10127 raft_consensus.cc:1081] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:47.765143  9928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41370
I20260709 12:08:47.765929  9928 raft_consensus.cc:493] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:47.766434  9928 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.767424 10004 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:08:47.769193 10176 consensus_queue.cc:1048] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.773391 10145 consensus_queue.cc:1048] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.775015  9928 raft_consensus.cc:515] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.777971  9928 leader_election.cc:290] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Requested vote from peers 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:47.778527 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.779772 10077 raft_consensus.cc:3055] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:47.780246 10077 raft_consensus.cc:740] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:47.783236 10002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:47.781590 10077 consensus_queue.cc:260] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.788045 10077 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.792523  9824 catalog_manager.cc:5697] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 1 to 2, leader changed from ed4d03d497fc443bb5c152dada8e137c (127.9.11.66) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 2 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.804976 10077 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:08:47.806366  9894 leader_election.cc:304] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: ed4d03d497fc443bb5c152dada8e137c
I20260709 12:08:47.807699 10125 raft_consensus.cc:2804] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:47.808243 10125 raft_consensus.cc:697] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:47.809120 10125 consensus_queue.cc:237] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:08:47.813380 10127 consensus_queue.cc:1048] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.823196 10163 consensus_queue.cc:1048] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.833217  9827 catalog_manager.cc:5697] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 1 to 2, leader changed from 36fe98593e134396a04a0350038d7791 (127.9.11.67) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 2 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.846369 10159 consensus_queue.cc:1048] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.854295 10125 consensus_queue.cc:1048] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.874760 10182 consensus_queue.cc:1048] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.884397 10134 raft_consensus.cc:993] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: : Instructing follower ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:08:47.884893 10134 raft_consensus.cc:1081] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Signalling peer ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:08:47.886485 10004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:58612
I20260709 12:08:47.887144 10004 raft_consensus.cc:493] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:47.887537 10004 raft_consensus.cc:3060] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.892994 10004 raft_consensus.cc:515] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.894603 10004 leader_election.cc:290] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 2 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:47.895800 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:47.895843  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41aa7a0f0fb74559be3977acd9d10dce" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:47.896872  9927 raft_consensus.cc:3055] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:47.896561 10078 raft_consensus.cc:3060] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.897281  9927 raft_consensus.cc:740] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:47.898207  9927 consensus_queue.cc:260] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.899585  9927 raft_consensus.cc:3060] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.904850 10078 raft_consensus.cc:2468] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 2.
I20260709 12:08:47.906205  9969 leader_election.cc:304] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:47.907104 10159 raft_consensus.cc:2804] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:47.907601 10159 raft_consensus.cc:697] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:47.907626  9927 raft_consensus.cc:2468] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 2.
I20260709 12:08:47.908600 10159 consensus_queue.cc:237] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:47.916215  9827 catalog_manager.cc:5697] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 1 to 2, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 2 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:48.005447 10127 consensus_queue.cc:1048] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:48.014268 10189 raft_consensus.cc:993] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:48.014868 10145 raft_consensus.cc:1081] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:08:48.017547  9928 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41370
I20260709 12:08:48.019306  9928 raft_consensus.cc:493] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:48.019793  9928 raft_consensus.cc:3060] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.021919 10145 consensus_queue.cc:1048] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:48.029356  9928 raft_consensus.cc:515] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:48.036448  9928 leader_election.cc:290] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:48.037844 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:48.038187 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84d8bfb3a4fc4f08a5be32940918d0c9" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:48.039242 10078 raft_consensus.cc:3055] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:48.039335 10004 raft_consensus.cc:3060] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.039757 10078 raft_consensus.cc:740] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:48.041254 10078 consensus_queue.cc:260] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:48.043596 10078 raft_consensus.cc:3060] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.050846 10004 raft_consensus.cc:2468] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:08:48.053251  9893 leader_election.cc:304] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:48.054507 10078 raft_consensus.cc:2468] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:08:48.054894 10182 raft_consensus.cc:2804] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:48.055785 10182 raft_consensus.cc:697] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:48.057250 10182 consensus_queue.cc:237] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:48.070214 10176 raft_consensus.cc:993] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: : Instructing follower ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:08:48.071010 10176 raft_consensus.cc:1081] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Signalling peer ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:08:48.073537 10004 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db68b5b7900243b090255540436c6791"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:58626
I20260709 12:08:48.073514  9827 catalog_manager.cc:5697] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 1 to 2, leader changed from 36fe98593e134396a04a0350038d7791 (127.9.11.67) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 2 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:48.074465 10004 raft_consensus.cc:493] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:48.075163 10004 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.087425 10004 raft_consensus.cc:515] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:48.090907 10004 leader_election.cc:290] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 2 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:08:48.091694  9928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:48.092197 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:08:48.092486  9928 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.093034 10078 raft_consensus.cc:3055] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:48.093513 10078 raft_consensus.cc:740] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:48.094776 10078 consensus_queue.cc:260] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:48.096773 10078 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.103478  9928 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 2.
I20260709 12:08:48.105250  9970 leader_election.cc:304] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:48.106392 10159 raft_consensus.cc:2804] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:48.106994 10159 raft_consensus.cc:697] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:08:48.108507 10078 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 2.
I20260709 12:08:48.108136 10159 consensus_queue.cc:237] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:48.117688  9827 catalog_manager.cc:5697] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 1 to 2, leader changed from 36fe98593e134396a04a0350038d7791 (127.9.11.67) to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 2 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:48.162652 10078 raft_consensus.cc:1275] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.164394 10182 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.174796 10004 raft_consensus.cc:1275] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.176432 10142 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.213687 10004 raft_consensus.cc:1275] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.214936 10142 consensus_queue.cc:1048] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.227187 10078 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.238933 10078 raft_consensus.cc:1275] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.240538 10125 consensus_queue.cc:1048] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.293097 10078 raft_consensus.cc:1275] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.294911 10125 consensus_queue.cc:1048] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.309254 10004 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.320086 10004 raft_consensus.cc:1275] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.321641 10125 consensus_queue.cc:1048] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.385152 10078 raft_consensus.cc:1275] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.386966 10161 consensus_queue.cc:1048] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:48.396903  9928 raft_consensus.cc:1275] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.399263 10163 consensus_queue.cc:1048] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:48.461508 10078 raft_consensus.cc:1275] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.463786 10134 consensus_queue.cc:1048] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:48.474769 10004 raft_consensus.cc:1275] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.477283 10142 consensus_queue.cc:1048] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.506394 10077 raft_consensus.cc:1275] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.507720 10212 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.517082  9928 raft_consensus.cc:1275] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.519083 10163 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:50.675715  9261 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:08:50.676632  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:50.698346  9928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24436744f48143448904edb1894af2e1"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:41392
I20260709 12:08:50.699091  9928 raft_consensus.cc:606] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Received request to transfer leadership to TS 36fe98593e134396a04a0350038d7791
I20260709 12:08:50.701783  9928 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:41392
I20260709 12:08:50.702322  9928 raft_consensus.cc:606] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Received request to transfer leadership to TS 36fe98593e134396a04a0350038d7791
I20260709 12:08:50.703511  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:08:50.704020  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:50.773311 10182 raft_consensus.cc:993] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: : Instructing follower 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:08:50.773900 10134 raft_consensus.cc:1081] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Signalling peer 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:08:50.775909 10078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24436744f48143448904edb1894af2e1"
dest_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:38724
I20260709 12:08:50.776541 10078 raft_consensus.cc:493] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:50.776923 10078 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:50.782853 10078 raft_consensus.cc:515] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:50.784868 10078 leader_election.cc:290] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 3 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:50.785792  9928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:50.785950 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:50.786398  9928 raft_consensus.cc:3055] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:50.786561 10004 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:50.786784  9928 raft_consensus.cc:740] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:50.787743  9928 consensus_queue.cc:260] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:50.788774  9928 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:50.791878 10004 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 3.
I20260709 12:08:50.793463 10042 leader_election.cc:304] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:50.794623  9928 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 3.
I20260709 12:08:50.794653 10145 raft_consensus.cc:2804] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:50.797000 10145 raft_consensus.cc:697] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:50.797907 10145 consensus_queue.cc:237] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:50.806180  9827 catalog_manager.cc:5697] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 2 to 3, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:50.845875 10182 raft_consensus.cc:993] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: : Instructing follower 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:08:50.846422 10182 raft_consensus.cc:1081] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Signalling peer 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:08:50.848138 10078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce"
dest_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:38724
I20260709 12:08:50.848767 10078 raft_consensus.cc:493] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:50.849164 10078 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.855427 10078 raft_consensus.cc:515] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:50.857525 10078 leader_election.cc:290] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 2 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:08:50.858445  9928 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:08:50.858675 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:08:50.859274 10004 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.859268  9928 raft_consensus.cc:3055] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:50.859807  9928 raft_consensus.cc:740] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:08:50.860643  9928 consensus_queue.cc:260] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:50.861737  9928 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.864271 10004 raft_consensus.cc:2468] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 2.
I20260709 12:08:50.865339 10042 leader_election.cc:304] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:08:50.866117 10145 raft_consensus.cc:2804] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:50.866587 10145 raft_consensus.cc:697] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:08:50.867573  9928 raft_consensus.cc:2468] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 2.
I20260709 12:08:50.867393 10145 consensus_queue.cc:237] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:08:50.875226  9827 catalog_manager.cc:5697] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 1 to 2, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:51.299214 10004 raft_consensus.cc:1275] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:51.300562 10145 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:51.311546  9928 raft_consensus.cc:1275] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:51.317582 10217 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:51.368532 10004 raft_consensus.cc:1275] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:51.370031 10145 consensus_queue.cc:1048] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:51.382027  9928 raft_consensus.cc:1275] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:51.383828 10227 consensus_queue.cc:1048] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:51.705323  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:51.711078  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:51.711606  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:52.712841  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:52.718801  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:52.719352  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:53.720386  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:53.727674  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:53.728178  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:54.729652  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:54.737323  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:54.737874  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:55.739214  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:55.744870  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:55.745376  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:56.746605  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:56.752285  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:56.752767  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:57.753870  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:57.759550  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:57.760035  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:58.761101  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:58.766674  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:58.767220  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:59.768430  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:59.774181  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:59.774756  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:00.811578  9261 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:09:00.822811 10077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce"
dest_uuid: "36fe98593e134396a04a0350038d7791"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:38738
I20260709 12:09:00.823691 10077 raft_consensus.cc:606] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:09:00.829636 10077 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24436744f48143448904edb1894af2e1"
dest_uuid: "36fe98593e134396a04a0350038d7791"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:38738
I20260709 12:09:00.830173 10077 raft_consensus.cc:606] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:09:00.838158 10004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:58632
I20260709 12:09:00.838671 10004 raft_consensus.cc:606] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:09:00.843688 10004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db68b5b7900243b090255540436c6791"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:58632
I20260709 12:09:00.844161 10004 raft_consensus.cc:606] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:09:00.848965 10004 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
mode: GRACEFUL
new_leader_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:58632
I20260709 12:09:00.849428 10004 raft_consensus.cc:606] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Received request to transfer leadership to TS a24b6211f48044f290dabea67c80438b
I20260709 12:09:00.940867 10242 raft_consensus.cc:993] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:00.941524 10246 raft_consensus.cc:1081] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:00.943352  9927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41384
I20260709 12:09:00.943823  9927 raft_consensus.cc:493] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:00.944121  9927 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:00.950539  9927 raft_consensus.cc:515] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:00.952482  9927 leader_election.cc:290] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:09:00.953662 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:00.953670 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:00.954636 10077 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:00.954682 10004 raft_consensus.cc:3055] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:00.955185 10004 raft_consensus.cc:740] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:09:00.956013 10004 consensus_queue.cc:260] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:00.957115 10004 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:00.961735 10077 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:09:00.962570 10004 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:09:00.963317  9894 leader_election.cc:304] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:09:00.964461 10238 raft_consensus.cc:2804] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:00.965866 10238 raft_consensus.cc:697] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:00.967044 10238 consensus_queue.cc:237] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:00.975669  9827 catalog_manager.cc:5697] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 1 to 2, leader changed from ed4d03d497fc443bb5c152dada8e137c (127.9.11.66) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 2 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.008486 10242 raft_consensus.cc:993] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.008997 10246 raft_consensus.cc:1081] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.010505  9927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db68b5b7900243b090255540436c6791"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41384
I20260709 12:09:01.011057  9927 raft_consensus.cc:493] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.011363  9927 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.017028  9927 raft_consensus.cc:515] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.018625  9927 leader_election.cc:290] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 3 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:09:01.019752 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:01.020097 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:01.020603 10004 raft_consensus.cc:3055] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:01.020726 10077 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.021075 10004 raft_consensus.cc:740] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:09:01.022100 10004 consensus_queue.cc:260] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.023554 10004 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.029358 10077 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 3.
I20260709 12:09:01.030959  9894 leader_election.cc:304] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:09:01.031786 10238 raft_consensus.cc:2804] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:01.031716 10004 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 3.
I20260709 12:09:01.032385 10238 raft_consensus.cc:697] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:01.033370 10238 consensus_queue.cc:237] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.042169  9827 catalog_manager.cc:5697] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 2 to 3, leader changed from ed4d03d497fc443bb5c152dada8e137c (127.9.11.66) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 3 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.044559 10240 raft_consensus.cc:993] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.045140 10240 raft_consensus.cc:1081] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.047899  9927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24436744f48143448904edb1894af2e1"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41370
I20260709 12:09:01.048491  9927 raft_consensus.cc:493] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.048869  9927 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:01.055259  9927 raft_consensus.cc:515] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.056818  9927 leader_election.cc:290] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 4 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:09:01.057840 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:01.058121 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24436744f48143448904edb1894af2e1" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:01.058480 10004 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:01.058804 10077 raft_consensus.cc:3055] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:09:01.059234 10077 raft_consensus.cc:740] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:09:01.060446 10077 consensus_queue.cc:260] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.061985 10077 raft_consensus.cc:3060] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:01.064723 10004 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 4.
I20260709 12:09:01.066222  9893 leader_election.cc:304] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:09:01.067201 10238 raft_consensus.cc:2804] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:09:01.067788 10238 raft_consensus.cc:697] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 4 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:01.068049 10077 raft_consensus.cc:2468] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 4.
I20260709 12:09:01.068739 10238 consensus_queue.cc:237] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.078223  9827 catalog_manager.cc:5697] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 3 to 4, leader changed from 36fe98593e134396a04a0350038d7791 (127.9.11.67) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 4 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.081182 10242 raft_consensus.cc:993] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.081764 10246 raft_consensus.cc:1081] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.083654  9927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41384
I20260709 12:09:01.084254  9927 raft_consensus.cc:493] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.084654  9927 raft_consensus.cc:3060] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:01.090829  9927 raft_consensus.cc:515] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.092635  9927 leader_election.cc:290] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:09:01.093683 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:01.093693 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76e37bffbed747f2b2280fbf995d7c0d" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:01.094370 10004 raft_consensus.cc:3055] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:01.094374 10077 raft_consensus.cc:3060] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:01.094650 10004 raft_consensus.cc:740] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:09:01.095440 10004 consensus_queue.cc:260] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.096433 10004 raft_consensus.cc:3060] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:01.099807 10077 raft_consensus.cc:2468] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:09:01.101115  9894 leader_election.cc:304] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, a24b6211f48044f290dabea67c80438b; no voters: 
I20260709 12:09:01.101845 10238 raft_consensus.cc:2804] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:01.101887 10004 raft_consensus.cc:2468] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 2.
I20260709 12:09:01.102387 10238 raft_consensus.cc:697] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:01.103329 10238 consensus_queue.cc:237] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.111285  9827 catalog_manager.cc:5697] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 1 to 2, leader changed from ed4d03d497fc443bb5c152dada8e137c (127.9.11.66) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 2 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.236658 10240 raft_consensus.cc:993] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: : Instructing follower a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.237406 10255 raft_consensus.cc:1081] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 LEADER]: Signalling peer a24b6211f48044f290dabea67c80438b to start an election
I20260709 12:09:01.239076  9927 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
 from {username='slave'} at 127.0.0.1:41370
I20260709 12:09:01.239655  9927 raft_consensus.cc:493] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.239991  9927 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.245241  9927 raft_consensus.cc:515] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.246887  9927 leader_election.cc:290] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 3 election: Requested vote from peers ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:09:01.248160 10004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:01.248172 10077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "988a4159073b4d23b82eb66372abc4ce" candidate_uuid: "a24b6211f48044f290dabea67c80438b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:01.248756 10004 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.248888 10077 raft_consensus.cc:3055] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:01.249241 10077 raft_consensus.cc:740] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:09:01.250088 10077 consensus_queue.cc:260] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.251345 10077 raft_consensus.cc:3060] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.254109 10004 raft_consensus.cc:2468] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 3.
I20260709 12:09:01.255244  9893 leader_election.cc:304] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a24b6211f48044f290dabea67c80438b, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:09:01.256120 10238 raft_consensus.cc:2804] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:01.256554 10238 raft_consensus.cc:697] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Becoming Leader. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:01.257617 10077 raft_consensus.cc:2468] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a24b6211f48044f290dabea67c80438b in term 3.
I20260709 12:09:01.257449 10238 consensus_queue.cc:237] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:01.265347  9827 catalog_manager.cc:5697] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b reported cstate change: term changed from 2 to 3, leader changed from 36fe98593e134396a04a0350038d7791 (127.9.11.67) to a24b6211f48044f290dabea67c80438b (127.9.11.65). New cstate: current_term: 3 leader_uuid: "a24b6211f48044f290dabea67c80438b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.384982 10004 raft_consensus.cc:1275] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:01.386377 10238 consensus_queue.cc:1048] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:01.398450 10077 raft_consensus.cc:1275] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:01.400183 10238 consensus_queue.cc:1048] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:01.453217 10004 raft_consensus.cc:1275] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.454591 10238 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:01.465721 10077 raft_consensus.cc:1275] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.468793 10238 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:01.618938 10004 raft_consensus.cc:1275] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:09:01.620397 10238 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:01.634296 10078 raft_consensus.cc:1275] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:09:01.635851 10238 consensus_queue.cc:1048] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:01.662763 10004 raft_consensus.cc:1275] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:01.664264 10238 consensus_queue.cc:1048] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:01.674336 10003 raft_consensus.cc:1275] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.676714 10238 consensus_queue.cc:1048] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:01.681243 10078 raft_consensus.cc:1275] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:01.683416 10265 consensus_queue.cc:1048] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:01.694082 10077 raft_consensus.cc:1275] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Refusing update from remote peer a24b6211f48044f290dabea67c80438b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.701514 10247 consensus_queue.cc:1048] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:03.888067  9261 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:09:03.888857  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:03.896544  9927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:41392
I20260709 12:09:03.897109  9927 raft_consensus.cc:606] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Received request to transfer leadership to TS ed4d03d497fc443bb5c152dada8e137c
I20260709 12:09:03.900161  9927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:41392
I20260709 12:09:03.900840  9927 raft_consensus.cc:606] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Received request to transfer leadership to TS 36fe98593e134396a04a0350038d7791
I20260709 12:09:03.903365  9927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db68b5b7900243b090255540436c6791"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:41392
I20260709 12:09:03.903849  9927 raft_consensus.cc:606] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Received request to transfer leadership to TS 36fe98593e134396a04a0350038d7791
I20260709 12:09:03.906279  9927 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e"
dest_uuid: "a24b6211f48044f290dabea67c80438b"
mode: GRACEFUL
new_leader_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:41392
I20260709 12:09:03.906810  9927 raft_consensus.cc:606] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Received request to transfer leadership to TS ed4d03d497fc443bb5c152dada8e137c
I20260709 12:09:03.907989  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:09:03.908701  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:03.943892 10247 raft_consensus.cc:993] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: : Instructing follower 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:09:03.944494 10248 raft_consensus.cc:1081] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Signalling peer 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:09:03.945979 10078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b"
dest_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:38724
I20260709 12:09:03.946444 10078 raft_consensus.cc:493] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:03.946681 10078 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:03.952344 10078 raft_consensus.cc:515] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:03.954627 10078 leader_election.cc:290] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 3 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:09:03.955662  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:09:03.955793 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf2f2ddecfd43aaa11525e193f5f09b" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:03.956305  9927 raft_consensus.cc:3055] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:03.956452 10003 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:03.956590  9927 raft_consensus.cc:740] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:03.957288  9927 consensus_queue.cc:260] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:03.958212  9927 raft_consensus.cc:3060] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:03.962357 10003 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 3.
I20260709 12:09:03.963099  9927 raft_consensus.cc:2468] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 3.
I20260709 12:09:03.963804 10042 leader_election.cc:304] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:09:03.964735 10273 raft_consensus.cc:2804] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:03.965253 10273 raft_consensus.cc:697] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:09:03.966095 10273 consensus_queue.cc:237] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:03.973489  9824 catalog_manager.cc:5697] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 2 to 3, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:04.114456 10247 raft_consensus.cc:993] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: : Instructing follower ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:09:04.114960 10248 raft_consensus.cc:1081] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Signalling peer ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:09:04.116617 10003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:58612
I20260709 12:09:04.117153 10003 raft_consensus.cc:493] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.117442 10003 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.122828 10003 raft_consensus.cc:515] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:04.125155 10003 leader_election.cc:290] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 3 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403)
I20260709 12:09:04.126322 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:04.126309  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552faba7084e4f3797c7a2ec992f6870" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:09:04.127101 10078 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.127166  9927 raft_consensus.cc:3055] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:04.127493  9927 raft_consensus.cc:740] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:04.128165  9927 consensus_queue.cc:260] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:04.129223  9927 raft_consensus.cc:3060] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.133481 10078 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 3.
I20260709 12:09:04.134256  9927 raft_consensus.cc:2468] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 3.
I20260709 12:09:04.134898  9969 leader_election.cc:304] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:09:04.135823 10246 raft_consensus.cc:2804] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:04.137399 10246 raft_consensus.cc:697] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 3 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:09:04.138797 10246 consensus_queue.cc:237] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:04.147639  9824 catalog_manager.cc:5697] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 2 to 3, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 3 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:04.319559 10247 raft_consensus.cc:993] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: : Instructing follower 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:09:04.320075 10248 raft_consensus.cc:1081] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Signalling peer 36fe98593e134396a04a0350038d7791 to start an election
I20260709 12:09:04.321689 10078 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db68b5b7900243b090255540436c6791"
dest_uuid: "36fe98593e134396a04a0350038d7791"
 from {username='slave'} at 127.0.0.1:38724
I20260709 12:09:04.322257 10078 raft_consensus.cc:493] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.322618 10078 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:04.329159 10078 raft_consensus.cc:515] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:04.331030 10078 leader_election.cc:290] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 4 election: Requested vote from peers a24b6211f48044f290dabea67c80438b (127.9.11.65:43595), ed4d03d497fc443bb5c152dada8e137c (127.9.11.66:40509)
I20260709 12:09:04.332100  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:09:04.332197 10003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db68b5b7900243b090255540436c6791" candidate_uuid: "36fe98593e134396a04a0350038d7791" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
I20260709 12:09:04.332831  9927 raft_consensus.cc:3055] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:09:04.332937 10003 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:04.333238  9927 raft_consensus.cc:740] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:04.334141  9927 consensus_queue.cc:260] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:04.335314  9927 raft_consensus.cc:3060] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:04.340077 10003 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 4.
I20260709 12:09:04.340910  9927 raft_consensus.cc:2468] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36fe98593e134396a04a0350038d7791 in term 4.
I20260709 12:09:04.341261 10042 leader_election.cc:304] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:09:04.342087 10273 raft_consensus.cc:2804] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:09:04.342540 10273 raft_consensus.cc:697] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 4 LEADER]: Becoming Leader. State: Replica: 36fe98593e134396a04a0350038d7791, State: Running, Role: LEADER
I20260709 12:09:04.343569 10273 consensus_queue.cc:237] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } }
I20260709 12:09:04.352485 10003 raft_consensus.cc:1275] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:04.353854 10273 consensus_queue.cc:1048] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:04.353390  9824 catalog_manager.cc:5697] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 reported cstate change: term changed from 3 to 4, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to 36fe98593e134396a04a0350038d7791 (127.9.11.67). New cstate: current_term: 4 leader_uuid: "36fe98593e134396a04a0350038d7791" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:04.369499  9927 raft_consensus.cc:1275] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:04.372442 10286 consensus_queue.cc:1048] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:04.456769 10247 raft_consensus.cc:993] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: : Instructing follower ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:09:04.457741 10248 raft_consensus.cc:1081] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Signalling peer ed4d03d497fc443bb5c152dada8e137c to start an election
I20260709 12:09:04.459874 10003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e"
dest_uuid: "ed4d03d497fc443bb5c152dada8e137c"
 from {username='slave'} at 127.0.0.1:58612
I20260709 12:09:04.460772 10003 raft_consensus.cc:493] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.461167 10003 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.468854 10003 raft_consensus.cc:515] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:09:04.471151 10003 leader_election.cc:290] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 3 election: Requested vote from peers 36fe98593e134396a04a0350038d7791 (127.9.11.67:44403), a24b6211f48044f290dabea67c80438b (127.9.11.65:43595)
I20260709 12:09:04.473420  9927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a24b6211f48044f290dabea67c80438b"
I20260709 12:09:04.473388 10078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd06b1b654243dabda6fb8210205d0e" candidate_uuid: "ed4d03d497fc443bb5c152dada8e137c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "36fe98593e134396a04a0350038d7791"
I20260709 12:09:04.474207  9927 raft_consensus.cc:3055] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:04.474331 10078 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.474586  9927 raft_consensus.cc:740] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a24b6211f48044f290dabea67c80438b, State: Running, Role: LEADER
I20260709 12:09:04.475598  9927 consensus_queue.cc:260] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:09:04.477036  9927 raft_consensus.cc:3060] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.481555 10078 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 3.
I20260709 12:09:04.483243  9969 leader_election.cc:304] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36fe98593e134396a04a0350038d7791, ed4d03d497fc443bb5c152dada8e137c; no voters: 
I20260709 12:09:04.484541 10246 raft_consensus.cc:2804] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:04.484555  9927 raft_consensus.cc:2468] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed4d03d497fc443bb5c152dada8e137c in term 3.
I20260709 12:09:04.485095 10246 raft_consensus.cc:697] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 3 LEADER]: Becoming Leader. State: Replica: ed4d03d497fc443bb5c152dada8e137c, State: Running, Role: LEADER
I20260709 12:09:04.486532 10246 consensus_queue.cc:237] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } }
I20260709 12:09:04.497588  9824 catalog_manager.cc:5697] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c reported cstate change: term changed from 2 to 3, leader changed from a24b6211f48044f290dabea67c80438b (127.9.11.65) to ed4d03d497fc443bb5c152dada8e137c (127.9.11.66). New cstate: current_term: 3 leader_uuid: "ed4d03d497fc443bb5c152dada8e137c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:04.694593 10078 raft_consensus.cc:1275] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:04.696134 10246 consensus_queue.cc:1048] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:04.699115  9927 raft_consensus.cc:1275] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:04.700570 10246 consensus_queue.cc:1048] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:04.850804 10003 raft_consensus.cc:1275] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:09:04.852360 10286 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed4d03d497fc443bb5c152dada8e137c" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:04.865273  9927 raft_consensus.cc:1275] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Refusing update from remote peer 36fe98593e134396a04a0350038d7791: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:09:04.867614 10286 consensus_queue.cc:1048] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:04.881453 10078 raft_consensus.cc:1275] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:04.882848 10246 consensus_queue.cc:1048] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "36fe98593e134396a04a0350038d7791" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:04.894309  9927 raft_consensus.cc:1275] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Refusing update from remote peer ed4d03d497fc443bb5c152dada8e137c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:04.895951 10246 consensus_queue.cc:1048] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24b6211f48044f290dabea67c80438b" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:04.910281  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:04.920099  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:04.920714  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:05.922269  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:05.929487  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:05.929972  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:06.931170  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:06.937129  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:06.937711  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:07.938815  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:07.944339  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:07.944763  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:08.945856  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:08.953436  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:08.953956  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:09.990111  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:09.995599  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:09.996100  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:09:10.890170  9858 debug-util.cc:398] Leaking SignalData structure 0x7b0800052d80 after lost signal to thread 9264
W20260709 12:09:10.891812  9858 debug-util.cc:398] Leaking SignalData structure 0x7b0800050460 after lost signal to thread 9861
W20260709 12:09:10.893237  9858 debug-util.cc:398] Leaking SignalData structure 0x7b08001ede40 after lost signal to thread 9953
W20260709 12:09:10.894456  9858 debug-util.cc:398] Leaking SignalData structure 0x7b0800159a60 after lost signal to thread 10028
W20260709 12:09:10.895677  9858 debug-util.cc:398] Leaking SignalData structure 0x7b0800193380 after lost signal to thread 10102
I20260709 12:09:10.997080  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:11.002861  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:11.003417  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:12.004510  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:12.011051  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:12.011602  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:13.125691  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:13.136373  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:13.137097  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:09:14.082265 10099 debug-util.cc:398] Leaking SignalData structure 0x7b08000a92a0 after lost signal to thread 9264
W20260709 12:09:14.083287 10099 debug-util.cc:398] Leaking SignalData structure 0x7b08000a9980 after lost signal to thread 9861
W20260709 12:09:14.084364 10099 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9ae0 after lost signal to thread 9953
W20260709 12:09:14.085295 10099 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1d80 after lost signal to thread 10028
W20260709 12:09:14.086185 10099 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7140 after lost signal to thread 10102
I20260709 12:09:14.180444  9261 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:09:14.181362  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
W20260709 12:09:14.195284  9970 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 12:09:14.200587  9970 consensus_peers.cc:597] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c -> Peer a24b6211f48044f290dabea67c80438b (127.9.11.65:43595): Couldn't send request to peer a24b6211f48044f290dabea67c80438b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:09:14.201479  9970 consensus_peers.cc:597] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c -> Peer a24b6211f48044f290dabea67c80438b (127.9.11.65:43595): Couldn't send request to peer a24b6211f48044f290dabea67c80438b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:14.234912  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:09:14.235946  9261 tablet_replica.cc:333] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.237037  9261 raft_consensus.cc:2243] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.238422  9261 raft_consensus.cc:2272] T 84d8bfb3a4fc4f08a5be32940918d0c9 P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.241812  9261 tablet_replica.cc:333] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.242799  9261 raft_consensus.cc:2243] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.243615  9261 raft_consensus.cc:2272] T 7cf2f2ddecfd43aaa11525e193f5f09b P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.246510  9261 tablet_replica.cc:333] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.247424  9261 raft_consensus.cc:2243] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.248013  9261 raft_consensus.cc:2272] T 552faba7084e4f3797c7a2ec992f6870 P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.250663  9261 tablet_replica.cc:333] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.251509  9261 raft_consensus.cc:2243] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.252710  9261 raft_consensus.cc:2272] T 76e37bffbed747f2b2280fbf995d7c0d P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.255576  9261 tablet_replica.cc:333] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.256403  9261 raft_consensus.cc:2243] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.257565  9261 raft_consensus.cc:2272] T 24436744f48143448904edb1894af2e1 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.260317  9261 tablet_replica.cc:333] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.261178  9261 raft_consensus.cc:2243] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.261817  9261 raft_consensus.cc:2272] T 41aa7a0f0fb74559be3977acd9d10dce P a24b6211f48044f290dabea67c80438b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.264739  9261 tablet_replica.cc:333] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.265631  9261 raft_consensus.cc:2243] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.266235  9261 raft_consensus.cc:2272] T db68b5b7900243b090255540436c6791 P a24b6211f48044f290dabea67c80438b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.269088  9261 tablet_replica.cc:333] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.269836  9261 raft_consensus.cc:2243] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.270383  9261 raft_consensus.cc:2272] T d025b065de3149af9dbe4e8665934bd1 P a24b6211f48044f290dabea67c80438b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.273191  9261 tablet_replica.cc:333] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.273895  9261 raft_consensus.cc:2243] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.274441  9261 raft_consensus.cc:2272] T fcd06b1b654243dabda6fb8210205d0e P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.277134  9261 tablet_replica.cc:333] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b: stopping tablet replica
I20260709 12:09:14.277815  9261 raft_consensus.cc:2243] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.279038  9261 raft_consensus.cc:2272] T 988a4159073b4d23b82eb66372abc4ce P a24b6211f48044f290dabea67c80438b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.320953  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
I20260709 12:09:14.351423  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
W20260709 12:09:14.383733 10044 consensus_peers.cc:597] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 -> Peer a24b6211f48044f290dabea67c80438b (127.9.11.65:43595): Couldn't send request to peer a24b6211f48044f290dabea67c80438b. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43595: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:14.396134  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:09:14.396927  9261 tablet_replica.cc:333] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.397745  9261 raft_consensus.cc:2243] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.398614  9261 raft_consensus.cc:2272] T fcd06b1b654243dabda6fb8210205d0e P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.400943  9261 tablet_replica.cc:333] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.401700  9261 raft_consensus.cc:2243] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.402215  9261 raft_consensus.cc:2272] T d025b065de3149af9dbe4e8665934bd1 P ed4d03d497fc443bb5c152dada8e137c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.404714  9261 tablet_replica.cc:333] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.405474  9261 raft_consensus.cc:2243] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.405992  9261 raft_consensus.cc:2272] T 7cf2f2ddecfd43aaa11525e193f5f09b P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.408102  9261 tablet_replica.cc:333] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.408658  9261 raft_consensus.cc:2243] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.409226  9261 raft_consensus.cc:2272] T 24436744f48143448904edb1894af2e1 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.411247  9261 tablet_replica.cc:333] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.411761  9261 raft_consensus.cc:2243] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.412237  9261 raft_consensus.cc:2272] T 988a4159073b4d23b82eb66372abc4ce P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.414306  9261 tablet_replica.cc:333] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.414814  9261 raft_consensus.cc:2243] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.415376  9261 raft_consensus.cc:2272] T db68b5b7900243b090255540436c6791 P ed4d03d497fc443bb5c152dada8e137c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.417325  9261 tablet_replica.cc:333] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.417825  9261 raft_consensus.cc:2243] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.418695  9261 raft_consensus.cc:2272] T 41aa7a0f0fb74559be3977acd9d10dce P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.420639  9261 tablet_replica.cc:333] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.421167  9261 raft_consensus.cc:2243] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.421602  9261 raft_consensus.cc:2272] T 84d8bfb3a4fc4f08a5be32940918d0c9 P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.423491  9261 tablet_replica.cc:333] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.444384  9261 raft_consensus.cc:2243] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.447925  9261 raft_consensus.cc:2272] T 76e37bffbed747f2b2280fbf995d7c0d P ed4d03d497fc443bb5c152dada8e137c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.451452  9261 tablet_replica.cc:333] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c: stopping tablet replica
I20260709 12:09:14.452234  9261 raft_consensus.cc:2243] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.454506  9261 raft_consensus.cc:2272] T 552faba7084e4f3797c7a2ec992f6870 P ed4d03d497fc443bb5c152dada8e137c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.489353  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
I20260709 12:09:14.525312  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
I20260709 12:09:14.564813  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:09:14.565893  9261 tablet_replica.cc:333] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.566790  9261 raft_consensus.cc:2243] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.567695  9261 raft_consensus.cc:2272] T 84d8bfb3a4fc4f08a5be32940918d0c9 P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.570466  9261 tablet_replica.cc:333] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.571219  9261 raft_consensus.cc:2243] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.571943  9261 raft_consensus.cc:2272] T 988a4159073b4d23b82eb66372abc4ce P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.574249  9261 tablet_replica.cc:333] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.574862  9261 raft_consensus.cc:2243] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.576550  9261 raft_consensus.cc:2272] T 7cf2f2ddecfd43aaa11525e193f5f09b P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.578949  9261 tablet_replica.cc:333] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.579555  9261 raft_consensus.cc:2243] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.581094  9261 raft_consensus.cc:2272] T d025b065de3149af9dbe4e8665934bd1 P 36fe98593e134396a04a0350038d7791 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.583395  9261 tablet_replica.cc:333] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.584002  9261 raft_consensus.cc:2243] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.584733  9261 raft_consensus.cc:2272] T fcd06b1b654243dabda6fb8210205d0e P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.587294  9261 tablet_replica.cc:333] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.587972  9261 raft_consensus.cc:2243] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.588640  9261 raft_consensus.cc:2272] T 552faba7084e4f3797c7a2ec992f6870 P 36fe98593e134396a04a0350038d7791 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.590641  9261 tablet_replica.cc:333] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.591287  9261 raft_consensus.cc:2243] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.591828  9261 raft_consensus.cc:2272] T 76e37bffbed747f2b2280fbf995d7c0d P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.594125  9261 tablet_replica.cc:333] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.594696  9261 raft_consensus.cc:2243] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.595417  9261 raft_consensus.cc:2272] T 24436744f48143448904edb1894af2e1 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.597692  9261 tablet_replica.cc:333] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.598215  9261 raft_consensus.cc:2243] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.598789  9261 raft_consensus.cc:2272] T 41aa7a0f0fb74559be3977acd9d10dce P 36fe98593e134396a04a0350038d7791 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.601125  9261 tablet_replica.cc:333] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791: stopping tablet replica
I20260709 12:09:14.601653  9261 raft_consensus.cc:2243] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.603155  9261 raft_consensus.cc:2272] T db68b5b7900243b090255540436c6791 P 36fe98593e134396a04a0350038d7791 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:09:15.638072  9261 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10099)
W20260709 12:09:16.638674  9261 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10099)
I20260709 12:09:16.725023  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:09:16.750571  9261 master.cc:562] Master@127.9.11.126:44893 shutting down...
I20260709 12:09:16.781138  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:09:16.781966  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:16.782363  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P a383cf8f549e4ebe990468b3cc2bb7c1: stopping tablet replica
I20260709 12:09:16.821960  9261 master.cc:584] Master@127.9.11.126:44893 shutdown complete.
36fe98593e134396a04a0350038d7791  1
a24b6211f48044f290dabea67c80438b  5
ed4d03d497fc443bb5c152dada8e137c  4
36fe98593e134396a04a0350038d7791  3
a24b6211f48044f290dabea67c80438b  3
ed4d03d497fc443bb5c152dada8e137c  4
36fe98593e134396a04a0350038d7791  1
a24b6211f48044f290dabea67c80438b  8
ed4d03d497fc443bb5c152dada8e137c  1
36fe98593e134396a04a0350038d7791  3
a24b6211f48044f290dabea67c80438b  4
ed4d03d497fc443bb5c152dada8e137c  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (31203 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:09:16.875499  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:42677
I20260709 12:09:16.876858  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:16.883728 10312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:16.883810 10311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:16.884119 10314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:16.886363  9261 server_base.cc:1061] running on GCE node
I20260709 12:09:16.887549  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:16.887739  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:16.887856  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598956887846 us; error 0 us; skew 500 ppm
I20260709 12:09:16.888406  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:16.891103  9261 webserver.cc:533] Webserver started at http://127.9.11.126:34213/ using document root <none> and password file <none>
I20260709 12:09:16.891605  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:16.891773  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:16.892024  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:16.893422  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "92c087afe23c499b9e821ff6f85b7e2a"
format_stamp: "Formatted at 2026-07-09 12:09:16 on dist-test-slave-k289"
I20260709 12:09:16.898499  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.007s
I20260709 12:09:16.902299 10319 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:16.903239  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:09:16.903512  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "92c087afe23c499b9e821ff6f85b7e2a"
format_stamp: "Formatted at 2026-07-09 12:09:16 on dist-test-slave-k289"
I20260709 12:09:16.903798  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:16.917354  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:16.918704  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:16.965348  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:42677
I20260709 12:09:16.965452 10370 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:42677 every 8 connection(s)
I20260709 12:09:16.970074 10371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.982649 10371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Bootstrap starting.
I20260709 12:09:16.988469 10371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.993673 10371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: No bootstrap required, opened a new log
I20260709 12:09:16.996357 10371 raft_consensus.cc:359] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c087afe23c499b9e821ff6f85b7e2a" member_type: VOTER }
I20260709 12:09:16.996865 10371 raft_consensus.cc:385] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.997149 10371 raft_consensus.cc:740] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c087afe23c499b9e821ff6f85b7e2a, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.997897 10371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c087afe23c499b9e821ff6f85b7e2a" member_type: VOTER }
I20260709 12:09:16.998450 10371 raft_consensus.cc:399] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:09:16.998699 10371 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:09:16.999022 10371 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.004467 10371 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c087afe23c499b9e821ff6f85b7e2a" member_type: VOTER }
I20260709 12:09:17.005092 10371 leader_election.cc:304] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92c087afe23c499b9e821ff6f85b7e2a; no voters: 
I20260709 12:09:17.006944 10371 leader_election.cc:290] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:09:17.007331 10374 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:17.009038 10374 raft_consensus.cc:697] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 1 LEADER]: Becoming Leader. State: Replica: 92c087afe23c499b9e821ff6f85b7e2a, State: Running, Role: LEADER
I20260709 12:09:17.009897 10374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c087afe23c499b9e821ff6f85b7e2a" member_type: VOTER }
I20260709 12:09:17.010931 10371 sys_catalog.cc:565] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:09:17.017208 10376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c087afe23c499b9e821ff6f85b7e2a. Latest consensus state: current_term: 1 leader_uuid: "92c087afe23c499b9e821ff6f85b7e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c087afe23c499b9e821ff6f85b7e2a" member_type: VOTER } }
I20260709 12:09:17.017908 10376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [sys.catalog]: This master's current role is: LEADER
I20260709 12:09:17.018054 10375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92c087afe23c499b9e821ff6f85b7e2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c087afe23c499b9e821ff6f85b7e2a" member_type: VOTER } }
I20260709 12:09:17.018633 10375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [sys.catalog]: This master's current role is: LEADER
I20260709 12:09:17.020445 10380 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:09:17.029593 10380 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:09:17.030717  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:09:17.040212 10380 catalog_manager.cc:1383] Generated new cluster ID: 74a9dd6aea7a48bfaeace693768c9341
I20260709 12:09:17.040566 10380 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:09:17.071353 10380 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:09:17.072853 10380 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:09:17.090879 10380 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Generated new TSK 0
I20260709 12:09:17.091651 10380 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:09:17.097435  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:17.104656 10392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:17.105669 10393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:17.107242 10395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:17.108439  9261 server_base.cc:1061] running on GCE node
I20260709 12:09:17.109625  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:17.109845  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:17.110038  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598957110016 us; error 0 us; skew 500 ppm
I20260709 12:09:17.110622  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:17.113509  9261 webserver.cc:533] Webserver started at http://127.9.11.65:44531/ using document root <none> and password file <none>
I20260709 12:09:17.114053  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:17.114269  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:17.114567  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:17.115865  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "99d42e7288f145f1bd5af212fd062995"
format_stamp: "Formatted at 2026-07-09 12:09:17 on dist-test-slave-k289"
I20260709 12:09:17.120971  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:09:17.124979 10400 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.125929  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:09:17.126269  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "99d42e7288f145f1bd5af212fd062995"
format_stamp: "Formatted at 2026-07-09 12:09:17 on dist-test-slave-k289"
I20260709 12:09:17.126557  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:17.141790  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:17.143383  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:17.145188  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:17.147838  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:17.148088  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.148365  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:17.148555  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.198839  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:42153
I20260709 12:09:17.198959 10462 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:42153 every 8 connection(s)
I20260709 12:09:17.204581  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:17.213001 10467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:17.214335 10468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:17.220050 10463 heartbeater.cc:344] Connected to a master server at 127.9.11.126:42677
I20260709 12:09:17.220464  9261 server_base.cc:1061] running on GCE node
I20260709 12:09:17.220691 10463 heartbeater.cc:461] Registering TS with master...
W20260709 12:09:17.221109 10471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:17.221801 10463 heartbeater.cc:507] Master 127.9.11.126:42677 requested a full tablet report, sending...
I20260709 12:09:17.223115  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:17.223410  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:17.223644  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598957223614 us; error 0 us; skew 500 ppm
I20260709 12:09:17.224527  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:17.224718 10336 ts_manager.cc:194] Registered new tserver with Master: 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:17.227967 10336 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46462
I20260709 12:09:17.228155  9261 webserver.cc:533] Webserver started at http://127.9.11.66:34739/ using document root <none> and password file <none>
I20260709 12:09:17.229001  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:17.229295  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:17.229686  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:17.231158  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
format_stamp: "Formatted at 2026-07-09 12:09:17 on dist-test-slave-k289"
I20260709 12:09:17.236217  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:09:17.240121 10475 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.241036  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:09:17.241426  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
format_stamp: "Formatted at 2026-07-09 12:09:17 on dist-test-slave-k289"
I20260709 12:09:17.241837  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:17.269397  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:17.270937  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:17.272814  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:17.275511  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:17.275753  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.276007  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:17.276191  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.324143  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:44681
I20260709 12:09:17.324277 10537 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:44681 every 8 connection(s)
I20260709 12:09:17.329523  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:17.338373 10541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:17.340215 10542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:17.343427 10544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:17.343675  9261 server_base.cc:1061] running on GCE node
I20260709 12:09:17.345335  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:17.345655  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:17.345712 10538 heartbeater.cc:344] Connected to a master server at 127.9.11.126:42677
I20260709 12:09:17.345846  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598957345828 us; error 0 us; skew 500 ppm
I20260709 12:09:17.346241 10538 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:17.346550  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:17.347237 10538 heartbeater.cc:507] Master 127.9.11.126:42677 requested a full tablet report, sending...
I20260709 12:09:17.349741 10336 ts_manager.cc:194] Registered new tserver with Master: 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:17.349961  9261 webserver.cc:533] Webserver started at http://127.9.11.67:43055/ using document root <none> and password file <none>
I20260709 12:09:17.350666  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:17.350930  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:17.351305  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:17.351812 10336 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46470
I20260709 12:09:17.353012  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "33d77e3063684586a737bccf358786d8"
format_stamp: "Formatted at 2026-07-09 12:09:17 on dist-test-slave-k289"
I20260709 12:09:17.357833  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:09:17.361490 10549 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.362282  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:09:17.362620  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "33d77e3063684586a737bccf358786d8"
format_stamp: "Formatted at 2026-07-09 12:09:17 on dist-test-slave-k289"
I20260709 12:09:17.362947  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:17.390195  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:17.391886  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:17.393685  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:17.396318  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:17.396566  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.396821  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:17.397013  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:17.447716  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:36727
I20260709 12:09:17.447850 10611 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:36727 every 8 connection(s)
I20260709 12:09:17.463889 10612 heartbeater.cc:344] Connected to a master server at 127.9.11.126:42677
I20260709 12:09:17.464359 10612 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:17.465301 10612 heartbeater.cc:507] Master 127.9.11.126:42677 requested a full tablet report, sending...
I20260709 12:09:17.467753 10336 ts_manager.cc:194] Registered new tserver with Master: 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:17.468616  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017170738s
I20260709 12:09:17.469293 10336 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46484
I20260709 12:09:17.498587 10336 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46490:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:09:17.505641 10336 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:09:17.734225 10577 tablet_service.cc:1511] Processing CreateTablet for tablet 58ffd928f77840c18d76b0b748c8ac26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:09:17.735911 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ffd928f77840c18d76b0b748c8ac26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.737577 10576 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8b322a475e40fcbca3445b6b0dcf26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:09:17.739107 10576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8b322a475e40fcbca3445b6b0dcf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.739178 10570 tablet_service.cc:1511] Processing CreateTablet for tablet 977ae790c1464057bdfdea6a1466fdc3 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:09:17.740736 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977ae790c1464057bdfdea6a1466fdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.743477 10572 tablet_service.cc:1511] Processing CreateTablet for tablet c5a50eeaf1ac462883e185eddec889f6 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:09:17.744369 10569 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:09:17.744817 10575 tablet_service.cc:1511] Processing CreateTablet for tablet 4642a95b220c4ecea8115de1fa98954e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:09:17.746730 10571 tablet_service.cc:1511] Processing CreateTablet for tablet c882875c377649acb233035d47b8132e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:09:17.749199 10574 tablet_service.cc:1511] Processing CreateTablet for tablet 773a46c59e194295b8f5f7c5b6d58216 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:09:17.750200 10503 tablet_service.cc:1511] Processing CreateTablet for tablet 58ffd928f77840c18d76b0b748c8ac26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:09:17.751462 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 63843cf9ccd4442cb52e3988597cabd8 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:09:17.752393 10502 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8b322a475e40fcbca3445b6b0dcf26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:09:17.751456 10573 tablet_service.cc:1511] Processing CreateTablet for tablet 4689962670ee421e8993be8a9ba7a657 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:09:17.752952 10498 tablet_service.cc:1511] Processing CreateTablet for tablet c5a50eeaf1ac462883e185eddec889f6 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:09:17.754043 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ffd928f77840c18d76b0b748c8ac26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.755363 10499 tablet_service.cc:1511] Processing CreateTablet for tablet 4689962670ee421e8993be8a9ba7a657 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:09:17.756817 10499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4689962670ee421e8993be8a9ba7a657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.756614 10500 tablet_service.cc:1511] Processing CreateTablet for tablet 773a46c59e194295b8f5f7c5b6d58216 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:09:17.758000 10500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773a46c59e194295b8f5f7c5b6d58216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.759193 10501 tablet_service.cc:1511] Processing CreateTablet for tablet 4642a95b220c4ecea8115de1fa98954e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:09:17.760651 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4642a95b220c4ecea8115de1fa98954e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.764678 10497 tablet_service.cc:1511] Processing CreateTablet for tablet c882875c377649acb233035d47b8132e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:09:17.765254 10496 tablet_service.cc:1511] Processing CreateTablet for tablet 977ae790c1464057bdfdea6a1466fdc3 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:09:17.763190 10502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8b322a475e40fcbca3445b6b0dcf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.766745 10496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977ae790c1464057bdfdea6a1466fdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.767464 10497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c882875c377649acb233035d47b8132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.768182 10498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a50eeaf1ac462883e185eddec889f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.771467 10428 tablet_service.cc:1511] Processing CreateTablet for tablet 58ffd928f77840c18d76b0b748c8ac26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:09:17.772878 10428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ffd928f77840c18d76b0b748c8ac26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.774495 10495 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:09:17.775877 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.778851 10427 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8b322a475e40fcbca3445b6b0dcf26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:09:17.780251 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8b322a475e40fcbca3445b6b0dcf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.782495 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63843cf9ccd4442cb52e3988597cabd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.784111 10573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4689962670ee421e8993be8a9ba7a657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.784212 10426 tablet_service.cc:1511] Processing CreateTablet for tablet 4642a95b220c4ecea8115de1fa98954e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:09:17.785633 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4642a95b220c4ecea8115de1fa98954e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.788288 10425 tablet_service.cc:1511] Processing CreateTablet for tablet 773a46c59e194295b8f5f7c5b6d58216 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:09:17.789651 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773a46c59e194295b8f5f7c5b6d58216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.785681 10569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.792654 10424 tablet_service.cc:1511] Processing CreateTablet for tablet 4689962670ee421e8993be8a9ba7a657 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:09:17.794164 10424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4689962670ee421e8993be8a9ba7a657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.796289 10423 tablet_service.cc:1511] Processing CreateTablet for tablet c5a50eeaf1ac462883e185eddec889f6 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:09:17.797499 10575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4642a95b220c4ecea8115de1fa98954e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.800329 10494 tablet_service.cc:1511] Processing CreateTablet for tablet 63843cf9ccd4442cb52e3988597cabd8 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:09:17.801723 10494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63843cf9ccd4442cb52e3988597cabd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.803399 10571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c882875c377649acb233035d47b8132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.806950 10574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773a46c59e194295b8f5f7c5b6d58216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.807238 10422 tablet_service.cc:1511] Processing CreateTablet for tablet c882875c377649acb233035d47b8132e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:09:17.810477 10421 tablet_service.cc:1511] Processing CreateTablet for tablet 977ae790c1464057bdfdea6a1466fdc3 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:09:17.811391 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a50eeaf1ac462883e185eddec889f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.797660 10423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a50eeaf1ac462883e185eddec889f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.814193 10420 tablet_service.cc:1511] Processing CreateTablet for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:09:17.826113 10421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 977ae790c1464057bdfdea6a1466fdc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.827361 10420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.830494 10422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c882875c377649acb233035d47b8132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.835693 10419 tablet_service.cc:1511] Processing CreateTablet for tablet 63843cf9ccd4442cb52e3988597cabd8 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:09:17.837255 10419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63843cf9ccd4442cb52e3988597cabd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.865446 10503 tablet_service.cc:1511] Processing CreateTablet for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:09:17.904182 10627 tablet_bootstrap.cc:492] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:17.907778 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.911588 10502 tablet_service.cc:1511] Processing CreateTablet for tablet 9a78d11ad8304de6b3c842374e1a2e09 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:09:17.919649 10576 tablet_service.cc:1511] Processing CreateTablet for tablet 9a78d11ad8304de6b3c842374e1a2e09 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:09:17.928368 10497 tablet_service.cc:1511] Processing CreateTablet for tablet 80de1e7bce9c4d34b0583315cafbc371 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:09:17.938863 10576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78d11ad8304de6b3c842374e1a2e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.948215 10421 tablet_service.cc:1511] Processing CreateTablet for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:09:17.949710 10421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.959703 10422 tablet_service.cc:1511] Processing CreateTablet for tablet 9a78d11ad8304de6b3c842374e1a2e09 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:09:17.960899 10424 tablet_service.cc:1511] Processing CreateTablet for tablet 80de1e7bce9c4d34b0583315cafbc371 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:09:17.967558 10629 tablet_bootstrap.cc:492] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:17.976958 10628 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:17.983642 10422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78d11ad8304de6b3c842374e1a2e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.994819 10500 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfb73cee8704df696d2eed4d27003bb (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:09:17.996980 10627 tablet_bootstrap.cc:654] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.993417 10420 tablet_service.cc:1511] Processing CreateTablet for tablet 26d3d17d58be40c8974c348ca146a951 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:09:17.997259 10496 tablet_service.cc:1511] Processing CreateTablet for tablet 26d3d17d58be40c8974c348ca146a951 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:09:17.998224 10420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d3d17d58be40c8974c348ca146a951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.000182 10425 tablet_service.cc:1511] Processing CreateTablet for tablet f5d55f0ab8c4445abbbca950ea76dca1 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:09:18.009516 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d55f0ab8c4445abbbca950ea76dca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.011552 10629 tablet_bootstrap.cc:654] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.002856 10426 tablet_service.cc:1511] Processing CreateTablet for tablet 859a2c4bf0aa4585ad9a84c0c4fbdb2d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:09:18.001256 10501 tablet_service.cc:1511] Processing CreateTablet for tablet 361eb2421d9548008ac1c2e32ba9ec14 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:09:18.013499 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859a2c4bf0aa4585ad9a84c0c4fbdb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.985641 10576 tablet_service.cc:1511] Processing CreateTablet for tablet 80de1e7bce9c4d34b0583315cafbc371 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:09:18.015919 10576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80de1e7bce9c4d34b0583315cafbc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.017331 10575 tablet_service.cc:1511] Processing CreateTablet for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:09:18.018690 10419 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfb73cee8704df696d2eed4d27003bb (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:09:18.019992 10419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfb73cee8704df696d2eed4d27003bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.021054 10577 tablet_service.cc:1511] Processing CreateTablet for tablet 26d3d17d58be40c8974c348ca146a951 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:09:18.018667 10575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:18.025440 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:18.026242 10628 tablet_bootstrap.cc:654] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.027297 10424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80de1e7bce9c4d34b0583315cafbc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.031345 10502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78d11ad8304de6b3c842374e1a2e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.032099 10497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80de1e7bce9c4d34b0583315cafbc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.035306 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361eb2421d9548008ac1c2e32ba9ec14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.039906 10423 tablet_service.cc:1511] Processing CreateTablet for tablet 361eb2421d9548008ac1c2e32ba9ec14 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:09:18.040460 10498 tablet_service.cc:1511] Processing CreateTablet for tablet b3872158fef542f0b0f902e3a8756feb (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:09:18.041282 10500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfb73cee8704df696d2eed4d27003bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.041445 10423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361eb2421d9548008ac1c2e32ba9ec14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.044126 10572 tablet_service.cc:1511] Processing CreateTablet for tablet 361eb2421d9548008ac1c2e32ba9ec14 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:09:18.045241 10498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3872158fef542f0b0f902e3a8756feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.050930 10496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d3d17d58be40c8974c348ca146a951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.052094 10428 tablet_service.cc:1511] Processing CreateTablet for tablet 9042bec741aa476587972e688997da43 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:09:18.053510 10428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042bec741aa476587972e688997da43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.059170 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d3d17d58be40c8974c348ca146a951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.070336 10629 tablet_bootstrap.cc:492] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:18.070926 10629 ts_tablet_manager.cc:1403] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.104s	user 0.014s	sys 0.010s
I20260709 12:09:18.071812 10499 tablet_service.cc:1511] Processing CreateTablet for tablet 9042bec741aa476587972e688997da43 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:09:18.074453 10499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042bec741aa476587972e688997da43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.074358 10629 raft_consensus.cc:359] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.075244 10629 raft_consensus.cc:385] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.075562 10629 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.076537 10629 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.095309 10570 tablet_service.cc:1511] Processing CreateTablet for tablet 9bfb73cee8704df696d2eed4d27003bb (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:09:18.097993 10503 tablet_service.cc:1511] Processing CreateTablet for tablet c7366ca68fce42f58f5e546ae5cad3ca (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:09:18.112036 10421 tablet_service.cc:1511] Processing CreateTablet for tablet b3872158fef542f0b0f902e3a8756feb (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:09:18.113597 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7366ca68fce42f58f5e546ae5cad3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.114236 10421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3872158fef542f0b0f902e3a8756feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.126951 10571 tablet_service.cc:1511] Processing CreateTablet for tablet 147a37ae746d4efb84fdc2f6d4d5981e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:09:18.119516 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361eb2421d9548008ac1c2e32ba9ec14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.139842 10569 tablet_service.cc:1511] Processing CreateTablet for tablet b3872158fef542f0b0f902e3a8756feb (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:09:18.140992 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bfb73cee8704df696d2eed4d27003bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.141764 10495 tablet_service.cc:1511] Processing CreateTablet for tablet 147a37ae746d4efb84fdc2f6d4d5981e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:09:18.143980 10574 tablet_service.cc:1511] Processing CreateTablet for tablet c7366ca68fce42f58f5e546ae5cad3ca (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:09:18.148703 10628 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:18.149278 10628 ts_tablet_manager.cc:1403] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.173s	user 0.029s	sys 0.031s
I20260709 12:09:18.156289 10427 tablet_service.cc:1511] Processing CreateTablet for tablet 147a37ae746d4efb84fdc2f6d4d5981e (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:09:18.157845 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a37ae746d4efb84fdc2f6d4d5981e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.159216 10494 tablet_service.cc:1511] Processing CreateTablet for tablet 6056d1f6a88d41d383e05a2c04508741 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:09:18.161094 10494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6056d1f6a88d41d383e05a2c04508741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.165544 10628 raft_consensus.cc:359] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.166399 10628 raft_consensus.cc:385] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.169564 10422 tablet_service.cc:1511] Processing CreateTablet for tablet c7366ca68fce42f58f5e546ae5cad3ca (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:09:18.174798 10422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7366ca68fce42f58f5e546ae5cad3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.175385 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a37ae746d4efb84fdc2f6d4d5981e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.166730 10628 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.179788 10628 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.184979 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 6056d1f6a88d41d383e05a2c04508741 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:09:18.191815 10463 heartbeater.cc:499] Master 127.9.11.126:42677 was elected leader, sending a full tablet report...
I20260709 12:09:18.191812 10633 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:18.203836 10613 tablet.cc:2406] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:09:18.208448 10629 ts_tablet_manager.cc:1434] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.005s
I20260709 12:09:18.204030 10612 heartbeater.cc:499] Master 127.9.11.126:42677 was elected leader, sending a full tablet report...
I20260709 12:09:18.211061 10627 tablet_bootstrap.cc:492] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:18.210907 10634 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.204257 10633 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.215346 10627 ts_tablet_manager.cc:1403] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.312s	user 0.063s	sys 0.048s
I20260709 12:09:18.215404 10634 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.218052 10633 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:18.220237 10628 ts_tablet_manager.cc:1434] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.009s
I20260709 12:09:18.220008 10629 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:18.221370 10628 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:18.218966 10627 raft_consensus.cc:359] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.222079 10627 raft_consensus.cc:385] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.174971 10573 tablet_service.cc:1511] Processing CreateTablet for tablet 9042bec741aa476587972e688997da43 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:09:18.201951 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6056d1f6a88d41d383e05a2c04508741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.223649 10425 tablet_service.cc:1511] Processing CreateTablet for tablet 6056d1f6a88d41d383e05a2c04508741 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:09:18.236371 10571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a37ae746d4efb84fdc2f6d4d5981e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.231508 10627 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.233170 10634 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:18.276121 10574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7366ca68fce42f58f5e546ae5cad3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.237378 10496 tablet_service.cc:1511] Processing CreateTablet for tablet a0bb77704fc9403481da02083315dbb1 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:09:18.283053 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6056d1f6a88d41d383e05a2c04508741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.291589 10569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3872158fef542f0b0f902e3a8756feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.282567 10627 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.304760 10426 tablet_service.cc:1511] Processing CreateTablet for tablet a0bb77704fc9403481da02083315dbb1 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:09:18.305815 10629 tablet_bootstrap.cc:654] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.306474 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0bb77704fc9403481da02083315dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.314085 10628 tablet_bootstrap.cc:654] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.318055 10420 tablet_service.cc:1511] Processing CreateTablet for tablet bea092b9128d46b6a42f4fe9aa23c739 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:09:18.319442 10573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042bec741aa476587972e688997da43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.327893 10496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0bb77704fc9403481da02083315dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.363350 10538 heartbeater.cc:499] Master 127.9.11.126:42677 was elected leader, sending a full tablet report...
I20260709 12:09:18.367380 10420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea092b9128d46b6a42f4fe9aa23c739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.375056 10575 tablet_service.cc:1511] Processing CreateTablet for tablet a0bb77704fc9403481da02083315dbb1 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:09:18.376787 10575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0bb77704fc9403481da02083315dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.304802 10419 tablet_service.cc:1511] Processing CreateTablet for tablet a082a11e47ba43b2acfaf230875b5e07 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:09:18.394086 10419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a082a11e47ba43b2acfaf230875b5e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.395838 10498 tablet_service.cc:1511] Processing CreateTablet for tablet f5d55f0ab8c4445abbbca950ea76dca1 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:09:18.404626 10498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d55f0ab8c4445abbbca950ea76dca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.404883 10499 tablet_service.cc:1511] Processing CreateTablet for tablet 859a2c4bf0aa4585ad9a84c0c4fbdb2d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:09:18.406497 10499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859a2c4bf0aa4585ad9a84c0c4fbdb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.404879 10576 tablet_service.cc:1511] Processing CreateTablet for tablet 859a2c4bf0aa4585ad9a84c0c4fbdb2d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:09:18.410130 10503 tablet_service.cc:1511] Processing CreateTablet for tablet a082a11e47ba43b2acfaf230875b5e07 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:09:18.455531 10577 tablet_service.cc:1511] Processing CreateTablet for tablet f5d55f0ab8c4445abbbca950ea76dca1 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:09:18.456856 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a082a11e47ba43b2acfaf230875b5e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.459780 10497 tablet_service.cc:1511] Processing CreateTablet for tablet bea092b9128d46b6a42f4fe9aa23c739 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:09:18.462136 10500 tablet_service.cc:1511] Processing CreateTablet for tablet 60f72c0bed7c45e484efbed50d2c364a (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:09:18.463627 10628 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:18.463788 10500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f72c0bed7c45e484efbed50d2c364a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.464210 10628 ts_tablet_manager.cc:1403] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.243s	user 0.019s	sys 0.017s
I20260709 12:09:18.468267 10628 raft_consensus.cc:359] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.469040 10628 raft_consensus.cc:385] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.469360 10628 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
W20260709 12:09:18.472052 10322 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 79191.1us
I20260709 12:09:18.473796 10502 tablet_service.cc:1511] Processing CreateTablet for tablet 4baf326f866a4a17b411837d7902a0c7 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:09:18.474416 10627 ts_tablet_manager.cc:1434] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.259s	user 0.011s	sys 0.006s
I20260709 12:09:18.475126 10576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 859a2c4bf0aa4585ad9a84c0c4fbdb2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.478629 10627 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:18.482115 10628 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.486542 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d55f0ab8c4445abbbca950ea76dca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.487229 10628 ts_tablet_manager.cc:1434] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.004s
I20260709 12:09:18.488497 10628 tablet_bootstrap.cc:492] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:18.498446 10628 tablet_bootstrap.cc:654] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.501324 10570 tablet_service.cc:1511] Processing CreateTablet for tablet a082a11e47ba43b2acfaf230875b5e07 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:09:18.504312 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a082a11e47ba43b2acfaf230875b5e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.515847 10577 tablet_service.cc:1511] Processing CreateTablet for tablet 60f72c0bed7c45e484efbed50d2c364a (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:09:18.517577 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f72c0bed7c45e484efbed50d2c364a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.517729 10572 tablet_service.cc:1511] Processing CreateTablet for tablet cfb1df1b6ba3414899a137b410511b26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:09:18.519223 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1df1b6ba3414899a137b410511b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.533365 10574 tablet_service.cc:1511] Processing CreateTablet for tablet 4baf326f866a4a17b411837d7902a0c7 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:09:18.538533 10569 tablet_service.cc:1511] Processing CreateTablet for tablet bea092b9128d46b6a42f4fe9aa23c739 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:09:18.556141 10574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4baf326f866a4a17b411837d7902a0c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.564783 10569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea092b9128d46b6a42f4fe9aa23c739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.569150 10576 tablet_service.cc:1511] Processing CreateTablet for tablet c545de01b7fb4c89946871c1d522006f (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:09:18.571309 10576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c545de01b7fb4c89946871c1d522006f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.611680 10424 tablet_service.cc:1511] Processing CreateTablet for tablet 60f72c0bed7c45e484efbed50d2c364a (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:09:18.612452 10629 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:18.616541 10629 ts_tablet_manager.cc:1403] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.397s	user 0.009s	sys 0.009s
I20260709 12:09:18.627274 10628 tablet_bootstrap.cc:492] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:18.627868 10424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f72c0bed7c45e484efbed50d2c364a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.628221 10628 ts_tablet_manager.cc:1403] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.140s	user 0.026s	sys 0.017s
I20260709 12:09:18.648494 10495 tablet_service.cc:1511] Processing CreateTablet for tablet c545de01b7fb4c89946871c1d522006f (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:09:18.632784 10628 raft_consensus.cc:359] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:18.659541 10628 raft_consensus.cc:385] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.659966 10628 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.660894 10628 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:18.662189 10501 tablet_service.cc:1511] Processing CreateTablet for tablet e4cd5d330f694dacb199bd61266c8f5a (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:09:18.663321 10628 ts_tablet_manager.cc:1434] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 12:09:18.664548 10628 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:18.623893 10629 raft_consensus.cc:359] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.667523 10629 raft_consensus.cc:385] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.667892 10629 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.485024 10497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea092b9128d46b6a42f4fe9aa23c739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.668877 10629 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.670110 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c545de01b7fb4c89946871c1d522006f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.672222 10425 tablet_service.cc:1511] Processing CreateTablet for tablet cfb1df1b6ba3414899a137b410511b26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:09:18.673770 10422 tablet_service.cc:1511] Processing CreateTablet for tablet 4baf326f866a4a17b411837d7902a0c7 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:09:18.674182 10628 tablet_bootstrap.cc:654] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.674672 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1df1b6ba3414899a137b410511b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.676699 10421 tablet_service.cc:1511] Processing CreateTablet for tablet c545de01b7fb4c89946871c1d522006f (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:09:18.678035 10421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c545de01b7fb4c89946871c1d522006f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.484593 10494 tablet_service.cc:1511] Processing CreateTablet for tablet cfb1df1b6ba3414899a137b410511b26 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:09:18.682499 10494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1df1b6ba3414899a137b410511b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.688225 10640 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.688966 10640 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.696436 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cd5d330f694dacb199bd61266c8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.707588 10629 ts_tablet_manager.cc:1434] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 12:09:18.710587 10629 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:18.713300 10573 tablet_service.cc:1511] Processing CreateTablet for tablet 51452865e973445d9e13f232cfb5efb9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:09:18.715346 10573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51452865e973445d9e13f232cfb5efb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.713320 10422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4baf326f866a4a17b411837d7902a0c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.791960 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:18.742266 10428 tablet_service.cc:1511] Processing CreateTablet for tablet e4cd5d330f694dacb199bd61266c8f5a (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:09:18.794759 10428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cd5d330f694dacb199bd61266c8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.761158 10420 tablet_service.cc:1511] Processing CreateTablet for tablet 51452865e973445d9e13f232cfb5efb9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:09:18.721356 10568 tablet_service.cc:1511] Processing CreateTablet for tablet e4cd5d330f694dacb199bd61266c8f5a (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:09:18.763495 10502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4baf326f866a4a17b411837d7902a0c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.799371 10496 tablet_service.cc:1511] Processing CreateTablet for tablet 51452865e973445d9e13f232cfb5efb9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:09:18.764398 10577 tablet_service.cc:1511] Processing CreateTablet for tablet f8b08b19bdbb4ac7b6bae9eccebb6335 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:09:18.800582 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cd5d330f694dacb199bd61266c8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.800678 10496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51452865e973445d9e13f232cfb5efb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.801654 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b08b19bdbb4ac7b6bae9eccebb6335. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:18.802248 10403 leader_election.cc:343] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.805464 10420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51452865e973445d9e13f232cfb5efb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.784993 10572 tablet_service.cc:1511] Processing CreateTablet for tablet 2e3622c19ce24dc4a8573de044a7656b (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:09:18.809154 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3622c19ce24dc4a8573de044a7656b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.809933 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:18.814026 10552 leader_election.cc:343] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.822314 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:18.824299 10634 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.719228 10640 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:18.836342 10499 tablet_service.cc:1511] Processing CreateTablet for tablet f8b08b19bdbb4ac7b6bae9eccebb6335 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:09:18.838189 10499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b08b19bdbb4ac7b6bae9eccebb6335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.824991 10634 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:18.848848 10503 tablet_service.cc:1511] Processing CreateTablet for tablet 2e3622c19ce24dc4a8573de044a7656b (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:09:18.850489 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3622c19ce24dc4a8573de044a7656b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.850817 10634 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:18.852187 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:18.853706 10552 leader_election.cc:343] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:18.863075 10403 leader_election.cc:343] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.863672 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:18.864856 10635 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:18.866313 10322 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82038.8us
I20260709 12:09:18.718508 10629 tablet_bootstrap.cc:654] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.875291 10427 tablet_service.cc:1511] Processing CreateTablet for tablet f8b08b19bdbb4ac7b6bae9eccebb6335 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:09:18.876734 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b08b19bdbb4ac7b6bae9eccebb6335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.878432 10575 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55454 (request call id 21) took 1015 ms. Trace:
I20260709 12:09:18.878907 10575 rpcz_store.cc:276] 0709 12:09:17.863407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:18.372987 (+509580us) service_pool.cc:224] Handling call
0709 12:09:18.376178 (+  3191us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:09:18.376297 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:09:18.551761 (+175464us) tablet_metadata.cc:665] Metadata flushed
0709 12:09:18.867148 (+315387us) tablet_replica.cc:206] Creating consensus instance
0709 12:09:18.878364 (+ 11216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:09:18.880640 10419 tablet_service.cc:1511] Processing CreateTablet for tablet 2e3622c19ce24dc4a8573de044a7656b (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:09:18.882079 10419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e3622c19ce24dc4a8573de044a7656b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:18.934056 10423 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.598s	user 0.014s	sys 0.025s
I20260709 12:09:18.930233 10627 tablet_bootstrap.cc:654] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.944041 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:18.948680 10424 tablet_service.cc:1511] Processing CreateTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:09:18.954008 10423 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48980 (request call id 15) took 1030 ms. Trace:
I20260709 12:09:18.956039 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:18.958859 10629 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:18.961012 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:18.971448 10424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deab11eaf55b4f3ebe48926a1e2afa38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.965517 10501 tablet_service.cc:1511] Processing CreateTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:09:18.973634 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deab11eaf55b4f3ebe48926a1e2afa38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.962029 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:18.972297 10479 leader_election.cc:343] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:18.972474 10553 leader_election.cc:343] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:18.979918 10478 leader_election.cc:343] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.980486 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:18.979853 10553 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:18.984189 10635 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.983961 10634 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.982290 10640 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.985882 10635 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:18.986948 10553 leader_election.cc:343] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.987465 10553 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:18.988062 10635 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:18.988222 10634 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.989398 10419 tablet_service.cc:1511] Processing CreateTablet for tablet b117a8eaf2494d0ea2317e5895cac532 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:09:18.989985 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:18.991024 10419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b117a8eaf2494d0ea2317e5895cac532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:18.991437 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:09:18.992915 10403 leader_election.cc:343] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:18.993633 10403 leader_election.cc:343] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.994014 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:18.995170 10635 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.969736 10423 rpcz_store.cc:276] 0709 12:09:17.923369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:18.028098 (+104729us) service_pool.cc:224] Handling call
0709 12:09:18.040916 (+ 12818us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:09:18.041035 (+   119us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:09:18.314756 (+273721us) tablet_metadata.cc:665] Metadata flushed
0709 12:09:18.953401 (+638645us) tablet_replica.cc:206] Creating consensus instance
0709 12:09:18.953933 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:09:18.984181 10629 ts_tablet_manager.cc:1403] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.274s	user 0.019s	sys 0.001s
I20260709 12:09:19.014017 10426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48980 (request call id 21) took 1087 ms. Trace:
I20260709 12:09:19.016490 10495 tablet_service.cc:1511] Processing CreateTablet for tablet b117a8eaf2494d0ea2317e5895cac532 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:09:19.017956 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b117a8eaf2494d0ea2317e5895cac532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.024962 10577 tablet_service.cc:1511] Processing CreateTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:09:19.026539 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deab11eaf55b4f3ebe48926a1e2afa38. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:19.028249 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:19.045243 10628 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:19.045892 10628 ts_tablet_manager.cc:1403] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.382s	user 0.059s	sys 0.038s
I20260709 12:09:19.048159 10572 tablet_service.cc:1511] Processing CreateTablet for tablet b117a8eaf2494d0ea2317e5895cac532 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:09:19.049341 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b117a8eaf2494d0ea2317e5895cac532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.049126 10628 raft_consensus.cc:359] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.049944 10628 raft_consensus.cc:385] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.050262 10628 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.053076 10628 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.055527 10628 ts_tablet_manager.cc:1434] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 12:09:19.057245 10628 tablet_bootstrap.cc:492] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:19.058307 10427 tablet_service.cc:1511] Processing CreateTablet for tablet 1ecefe266eea48c4af129b425f391fe9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:09:19.014463 10426 rpcz_store.cc:276] 0709 12:09:17.926458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:18.303569 (+377111us) service_pool.cc:224] Handling call
0709 12:09:18.305805 (+  2236us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:09:18.306016 (+   211us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:09:18.741426 (+435410us) tablet_metadata.cc:665] Metadata flushed
0709 12:09:19.010708 (+269282us) tablet_replica.cc:206] Creating consensus instance
0709 12:09:19.013945 (+  3237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:09:19.060088 10634 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.060598 10634 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:19.063263 10627 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:19.063817 10627 ts_tablet_manager.cc:1403] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.585s	user 0.029s	sys 0.012s
I20260709 12:09:19.064213 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.066542 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.070108 10634 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.059754 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ecefe266eea48c4af129b425f391fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.071929 10426 tablet_service.cc:1511] Processing CreateTablet for tablet 05e004719a9442e2b9b045e4a6df5426 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:09:19.073321 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05e004719a9442e2b9b045e4a6df5426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.075615 10628 tablet_bootstrap.cc:654] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.055306 10629 raft_consensus.cc:359] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.067055 10627 raft_consensus.cc:359] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:19.068480 10552 leader_election.cc:343] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.076162 10627 raft_consensus.cc:385] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:09:19.076565 10553 leader_election.cc:343] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.076681 10627 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.077665 10627 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.079080 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.079809 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 1ecefe266eea48c4af129b425f391fe9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:09:19.080272 10634 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.081475 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ecefe266eea48c4af129b425f391fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.082090 10503 tablet_service.cc:1511] Processing CreateTablet for tablet 1ecefe266eea48c4af129b425f391fe9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:09:19.083437 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ecefe266eea48c4af129b425f391fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.086082 10628 tablet_bootstrap.cc:492] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:19.086603 10628 ts_tablet_manager.cc:1403] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 12:09:19.088415 10627 ts_tablet_manager.cc:1434] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.001s
I20260709 12:09:19.089849 10628 raft_consensus.cc:359] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.090619 10628 raft_consensus.cc:385] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.090915 10628 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.091991 10628 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.094349 10628 ts_tablet_manager.cc:1434] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:09:19.095351 10627 tablet_bootstrap.cc:492] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:19.095422 10628 tablet_bootstrap.cc:492] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:19.076153 10629 raft_consensus.cc:385] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.102680 10501 tablet_service.cc:1511] Processing CreateTablet for tablet 05e004719a9442e2b9b045e4a6df5426 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:09:19.109768 10629 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.110513 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05e004719a9442e2b9b045e4a6df5426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.121662 10420 tablet_service.cc:1511] Processing CreateTablet for tablet a80f9fe1a0f840c4a6b9984e5fc64eed (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:09:19.123287 10420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f9fe1a0f840c4a6b9984e5fc64eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.126719 10628 tablet_bootstrap.cc:654] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.129032 10634 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.142900 10634 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.144093 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 7919194a8a8542559be104753f043851 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:09:19.131999 10424 tablet_service.cc:1511] Processing CreateTablet for tablet 7919194a8a8542559be104753f043851 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:09:19.128518 10627 tablet_bootstrap.cc:654] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.150328 10424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7919194a8a8542559be104753f043851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.134727 10495 tablet_service.cc:1511] Processing CreateTablet for tablet a80f9fe1a0f840c4a6b9984e5fc64eed (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:09:19.135586 10425 tablet_service.cc:1511] Processing CreateTablet for tablet 66985e55da4c4d3e88491e146eaaded2 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:09:19.153053 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66985e55da4c4d3e88491e146eaaded2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.156291 10426 tablet_service.cc:1511] Processing CreateTablet for tablet dc98cd5edc9240fba53978ed59191cbd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:09:19.157766 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc98cd5edc9240fba53978ed59191cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.136719 10629 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.157766 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7919194a8a8542559be104753f043851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.160194 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.160998 10629 ts_tablet_manager.cc:1434] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.137s	user 0.007s	sys 0.000s
I20260709 12:09:19.161936 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f9fe1a0f840c4a6b9984e5fc64eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.162128 10629 tablet_bootstrap.cc:492] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:19.137296 10572 tablet_service.cc:1511] Processing CreateTablet for tablet a80f9fe1a0f840c4a6b9984e5fc64eed (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:09:19.163820 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a80f9fe1a0f840c4a6b9984e5fc64eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.138393 10496 tablet_service.cc:1511] Processing CreateTablet for tablet 7919194a8a8542559be104753f043851 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:09:19.170686 10496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7919194a8a8542559be104753f043851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.141609 10499 tablet_service.cc:1511] Processing CreateTablet for tablet 66985e55da4c4d3e88491e146eaaded2 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:09:19.173197 10499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66985e55da4c4d3e88491e146eaaded2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:19.175415 10552 leader_election.cc:343] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.145231 10634 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.146976 10423 tablet_service.cc:1511] Processing CreateTablet for tablet 955c0a21e1e04f51bbbc7c06b5d611f5 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:09:19.180725 10423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955c0a21e1e04f51bbbc7c06b5d611f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.181456 10427 tablet_service.cc:1511] Processing CreateTablet for tablet b4d2e9ee3d824817989d1fa5abcd9d67 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:09:19.148801 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.182796 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d2e9ee3d824817989d1fa5abcd9d67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:19.183884 10553 leader_election.cc:343] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.184456 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.185673 10634 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.128608 10577 tablet_service.cc:1511] Processing CreateTablet for tablet 05e004719a9442e2b9b045e4a6df5426 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:09:19.204037 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05e004719a9442e2b9b045e4a6df5426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.204893 10628 tablet_bootstrap.cc:492] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:19.205507 10628 ts_tablet_manager.cc:1403] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.110s	user 0.018s	sys 0.006s
I20260709 12:09:19.208937 10628 raft_consensus.cc:359] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.209525 10628 raft_consensus.cc:385] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.209806 10628 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.210485 10628 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.212368 10628 ts_tablet_manager.cc:1434] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:19.213450 10628 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:19.215618 10503 tablet_service.cc:1511] Processing CreateTablet for tablet 955c0a21e1e04f51bbbc7c06b5d611f5 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:09:19.216539 10574 tablet_service.cc:1511] Processing CreateTablet for tablet 955c0a21e1e04f51bbbc7c06b5d611f5 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:09:19.217051 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955c0a21e1e04f51bbbc7c06b5d611f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.218765 10570 tablet_service.cc:1511] Processing CreateTablet for tablet dc98cd5edc9240fba53978ed59191cbd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:09:19.220494 10494 tablet_service.cc:1511] Processing CreateTablet for tablet b4d2e9ee3d824817989d1fa5abcd9d67 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:09:19.221764 10629 tablet_bootstrap.cc:654] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.222338 10574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955c0a21e1e04f51bbbc7c06b5d611f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.223004 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc98cd5edc9240fba53978ed59191cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.221774 10494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d2e9ee3d824817989d1fa5abcd9d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.233389 10501 tablet_service.cc:1511] Processing CreateTablet for tablet dc98cd5edc9240fba53978ed59191cbd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:09:19.239867 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc98cd5edc9240fba53978ed59191cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.235906 10568 tablet_service.cc:1511] Processing CreateTablet for tablet 66985e55da4c4d3e88491e146eaaded2 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:09:19.242169 10628 tablet_bootstrap.cc:654] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.243054 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66985e55da4c4d3e88491e146eaaded2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.246134 10502 tablet_service.cc:1511] Processing CreateTablet for tablet 06d2724d97aa43298ebaf9e746580fd0 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:09:19.247663 10502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d2724d97aa43298ebaf9e746580fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.248613 10574 tablet_service.cc:1511] Processing CreateTablet for tablet b4d2e9ee3d824817989d1fa5abcd9d67 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:09:19.253227 10573 tablet_service.cc:1511] Processing CreateTablet for tablet 06d2724d97aa43298ebaf9e746580fd0 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:09:19.254211 10628 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:19.256163 10628 ts_tablet_manager.cc:1403] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.009s
I20260709 12:09:19.256382 10427 tablet_service.cc:1511] Processing CreateTablet for tablet 06d2724d97aa43298ebaf9e746580fd0 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:09:19.257987 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d2724d97aa43298ebaf9e746580fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.259400 10574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d2e9ee3d824817989d1fa5abcd9d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.262023 10573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d2724d97aa43298ebaf9e746580fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.274786 10426 tablet_service.cc:1511] Processing CreateTablet for tablet 448b86aa128243e48bed5f471ada874f (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:09:19.277714 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448b86aa128243e48bed5f471ada874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.280192 10628 raft_consensus.cc:359] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.280992 10628 raft_consensus.cc:385] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.281320 10628 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.282141 10628 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.288030 10497 tablet_service.cc:1511] Processing CreateTablet for tablet 448b86aa128243e48bed5f471ada874f (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:09:19.289561 10497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448b86aa128243e48bed5f471ada874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.292968 10570 tablet_service.cc:1511] Processing CreateTablet for tablet 448b86aa128243e48bed5f471ada874f (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:09:19.294494 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448b86aa128243e48bed5f471ada874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.294343 10423 tablet_service.cc:1511] Processing CreateTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:09:19.295744 10423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b474d9b241d4a698702287ee1bbe3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.307194 10627 tablet_bootstrap.cc:492] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:19.307901 10627 ts_tablet_manager.cc:1403] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.011s
I20260709 12:09:19.312776 10627 raft_consensus.cc:359] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:19.313655 10627 raft_consensus.cc:385] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.314109 10627 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.315080 10629 tablet_bootstrap.cc:492] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:19.323508 10629 ts_tablet_manager.cc:1403] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.162s	user 0.016s	sys 0.000s
I20260709 12:09:19.324116 10627 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:19.326951 10629 raft_consensus.cc:359] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.330238 10629 raft_consensus.cc:385] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.332526 10628 ts_tablet_manager.cc:1434] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.001s
I20260709 12:09:19.332815 10503 tablet_service.cc:1511] Processing CreateTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:09:19.333807 10628 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:19.334255 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b474d9b241d4a698702287ee1bbe3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.337786 10572 tablet_service.cc:1511] Processing CreateTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:09:19.338366 10629 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.339368 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b474d9b241d4a698702287ee1bbe3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.339442 10629 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.340978 10627 ts_tablet_manager.cc:1434] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:09:19.350633 10640 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.351275 10640 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.353749 10628 tablet_bootstrap.cc:654] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.353837 10629 ts_tablet_manager.cc:1434] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:09:19.355373 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:19.356073 10587 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:19.357728 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.360605 10478 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:19.362469 10627 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
W20260709 12:09:19.362725 10479 leader_election.cc:343] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.363863 10640 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:19.364444 10640 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.364902 10640 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.365281 10640 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.372299 10425 tablet_service.cc:1511] Processing CreateTablet for tablet f94d710c36f04a9b8685258a7ff9d8a0 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:09:19.373903 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94d710c36f04a9b8685258a7ff9d8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.373538 10494 tablet_service.cc:1511] Processing CreateTablet for tablet f94d710c36f04a9b8685258a7ff9d8a0 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:09:19.374745 10629 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:19.374943 10494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94d710c36f04a9b8685258a7ff9d8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.380110 10570 tablet_service.cc:1511] Processing CreateTablet for tablet f94d710c36f04a9b8685258a7ff9d8a0 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:09:19.381526 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f94d710c36f04a9b8685258a7ff9d8a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.382696 10427 tablet_service.cc:1511] Processing CreateTablet for tablet 31117e61be004c52a161168a559023fd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:09:19.384136 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31117e61be004c52a161168a559023fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.389173 10634 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.389559 10501 tablet_service.cc:1511] Processing CreateTablet for tablet 31117e61be004c52a161168a559023fd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:09:19.389736 10634 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.391037 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31117e61be004c52a161168a559023fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.393448 10634 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.393718 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.394380 10436 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:19.393352 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.395702 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:19.396504 10634 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.397027 10634 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.397410 10634 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:19.396502 10552 leader_election.cc:343] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.398702 10627 tablet_bootstrap.cc:654] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.408450 10572 tablet_service.cc:1511] Processing CreateTablet for tablet 945dd7b821a5493f8a9d6ca251795da9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:09:19.399507 10640 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.402179 10629 tablet_bootstrap.cc:654] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.409897 10572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945dd7b821a5493f8a9d6ca251795da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.404675 10628 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:19.405864 10502 tablet_service.cc:1511] Processing CreateTablet for tablet 945dd7b821a5493f8a9d6ca251795da9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:09:19.412082 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:19.412587 10502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945dd7b821a5493f8a9d6ca251795da9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:19.413612 10479 leader_election.cc:343] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.416123 10497 tablet_service.cc:1511] Processing CreateTablet for tablet e8daad82d9f346a295fe1cf354383acc (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:09:19.416486 10634 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.417769 10497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8daad82d9f346a295fe1cf354383acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.418116 10640 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:19.418788 10628 ts_tablet_manager.cc:1403] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.004s
I20260709 12:09:19.400086 10573 tablet_service.cc:1511] Processing CreateTablet for tablet 31117e61be004c52a161168a559023fd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:09:19.408608 10426 tablet_service.cc:1511] Processing CreateTablet for tablet e8daad82d9f346a295fe1cf354383acc (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:09:19.404538 10423 tablet_service.cc:1511] Processing CreateTablet for tablet 945dd7b821a5493f8a9d6ca251795da9 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:09:19.423278 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:19.426506 10634 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.426698 10426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8daad82d9f346a295fe1cf354383acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.427266 10634 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.427328 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:19.426595 10587 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.428114 10424 tablet_service.cc:1511] Processing CreateTablet for tablet cc34ac39d2184bffb94d60fe15003395 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:09:19.429479 10424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc34ac39d2184bffb94d60fe15003395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.428000 10436 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.427836 10634 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.446040 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:19.443141 10568 tablet_service.cc:1511] Processing CreateTablet for tablet e8daad82d9f346a295fe1cf354383acc (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:09:19.445994 10420 tablet_service.cc:1511] Processing CreateTablet for tablet 38ead18e1a814c8b958f80396faa3170 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
W20260709 12:09:19.447726 10552 leader_election.cc:343] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.444990 10628 raft_consensus.cc:359] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.452978 10628 raft_consensus.cc:385] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.453388 10628 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.453661 10420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ead18e1a814c8b958f80396faa3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.450570 10573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31117e61be004c52a161168a559023fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.455535 10574 tablet_service.cc:1511] Processing CreateTablet for tablet cc34ac39d2184bffb94d60fe15003395 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:09:19.456910 10503 tablet_service.cc:1511] Processing CreateTablet for tablet cc34ac39d2184bffb94d60fe15003395 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:09:19.457937 10628 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.458949 10503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc34ac39d2184bffb94d60fe15003395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.460259 10496 tablet_service.cc:1511] Processing CreateTablet for tablet 38ead18e1a814c8b958f80396faa3170 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:09:19.461623 10496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ead18e1a814c8b958f80396faa3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.464181 10577 tablet_service.cc:1511] Processing CreateTablet for tablet 38ead18e1a814c8b958f80396faa3170 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:09:19.470175 10568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8daad82d9f346a295fe1cf354383acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.469803 10499 tablet_service.cc:1511] Processing CreateTablet for tablet e81a69eea26d461e9662b9b2aa4151cd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:09:19.473178 10574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc34ac39d2184bffb94d60fe15003395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.474092 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ead18e1a814c8b958f80396faa3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.477782 10628 ts_tablet_manager.cc:1434] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 12:09:19.486138 10425 tablet_service.cc:1511] Processing CreateTablet for tablet e81a69eea26d461e9662b9b2aa4151cd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:09:19.474092 10499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81a69eea26d461e9662b9b2aa4151cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.525112 10425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81a69eea26d461e9662b9b2aa4151cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.521598 10635 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.527882 10569 tablet_service.cc:1511] Processing CreateTablet for tablet e81a69eea26d461e9662b9b2aa4151cd (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:09:19.527906 10635 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.535261 10569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81a69eea26d461e9662b9b2aa4151cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.558450 10660 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.559316 10660 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.574321 10423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945dd7b821a5493f8a9d6ca251795da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.589335 10635 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:19.589212 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.590098 10635 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.590224 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:19.590677 10635 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:19.591262 10403 leader_election.cc:343] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.593776 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.594498 10513 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:19.597178 10427 tablet_service.cc:1511] Processing CreateTablet for tablet c61f4bfa2db8497dbf7e0446e2ab707d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:09:19.597527 10587 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:19.598915 10478 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.600334 10640 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.600045 10495 tablet_service.cc:1511] Processing CreateTablet for tablet c61f4bfa2db8497dbf7e0446e2ab707d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:09:19.600880 10640 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:19.601539 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61f4bfa2db8497dbf7e0446e2ab707d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.601854 10640 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.604164 10576 tablet_service.cc:1511] Processing CreateTablet for tablet c61f4bfa2db8497dbf7e0446e2ab707d (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:09:19.604871 10634 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.605726 10576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61f4bfa2db8497dbf7e0446e2ab707d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.607357 10660 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.624023 10628 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:19.639230 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.639528 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.640020 10500 tablet_service.cc:1511] Processing CreateTablet for tablet 81b70ca03fbe4622a03717efd079d598 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 12:09:19.641480 10552 leader_election.cc:343] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.640407 10635 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:19.636610 10419 tablet_service.cc:1511] Processing CreateTablet for tablet 81b70ca03fbe4622a03717efd079d598 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:09:19.647081 10660 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.647593 10660 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:19.642300 10552 leader_election.cc:343] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.649201 10336 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:19.651090 10660 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.654192 10575 tablet_service.cc:1511] Processing CreateTablet for tablet 81b70ca03fbe4622a03717efd079d598 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:09:19.655128 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61f4bfa2db8497dbf7e0446e2ab707d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.656108 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.658015 10419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b70ca03fbe4622a03717efd079d598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.656414 10628 tablet_bootstrap.cc:654] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.641472 10500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b70ca03fbe4622a03717efd079d598. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:19.659392 10552 leader_election.cc:343] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.674782 10575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b70ca03fbe4622a03717efd079d598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.687616 10422 tablet_service.cc:1511] Processing CreateTablet for tablet 5613e87b5a8c4948ae15672ea77f31cf (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:09:19.689145 10422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5613e87b5a8c4948ae15672ea77f31cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.701709 10436 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:19.703444 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.708200 10640 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.711738 10640 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:19.713663 10629 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:19.714246 10629 ts_tablet_manager.cc:1403] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.340s	user 0.015s	sys 0.011s
I20260709 12:09:19.715548 10627 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:19.716096 10627 ts_tablet_manager.cc:1403] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.354s	user 0.016s	sys 0.025s
I20260709 12:09:19.714143 10640 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:19.716485 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:19.717199 10587 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:19.718261 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:19.719853 10479 leader_election.cc:343] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.720510 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.721859 10640 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.722246 10640 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.722589 10640 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:19.725975 10608 debug-util.cc:398] Leaking SignalData structure 0x7b080013a940 after lost signal to thread 9264
I20260709 12:09:19.726356 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:19.727156 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08001a4a40 after lost signal to thread 10370
I20260709 12:09:19.726851 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:19.728924 10635 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:09:19.728255 10403 leader_election.cc:343] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:19.729434 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08000e19e0 after lost signal to thread 10537
I20260709 12:09:19.729810 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:09:19.730618 10608 debug-util.cc:398] Leaking SignalData structure 0x7b0800117080 after lost signal to thread 10611
I20260709 12:09:19.729470 10635 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.731428 10635 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.737883 10640 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:19.739543 10663 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.740605 10635 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.739838 10495 tablet_service.cc:1511] Processing CreateTablet for tablet 5613e87b5a8c4948ae15672ea77f31cf (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:09:19.742199 10495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5613e87b5a8c4948ae15672ea77f31cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.745205 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:19.745960 10587 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.754629 10570 tablet_service.cc:1511] Processing CreateTablet for tablet 5613e87b5a8c4948ae15672ea77f31cf (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:09:19.756255 10570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5613e87b5a8c4948ae15672ea77f31cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.762024 10640 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:19.762553 10421 tablet_service.cc:1511] Processing CreateTablet for tablet d62573c8e5a64ece95bc46d5fa18185c (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:09:19.765476 10421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d62573c8e5a64ece95bc46d5fa18185c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.763335 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:19.766525 10629 raft_consensus.cc:359] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.767321 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.768224 10627 raft_consensus.cc:359] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:19.768692 10479 leader_election.cc:343] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.769034 10627 raft_consensus.cc:385] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.760407 10494 tablet_service.cc:1511] Processing CreateTablet for tablet d62573c8e5a64ece95bc46d5fa18185c (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:09:19.769353 10627 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.769699 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:19.770401 10627 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.771414 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:19.772786 10660 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.773336 10660 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:19.774272 10660 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:19.775525 10553 leader_election.cc:343] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.764158 10587 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:19.776032 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.777710 10634 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.780325 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.781522 10658 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.772925 10627 ts_tablet_manager.cc:1434] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 12:09:19.787689 10628 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:19.792748 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:19.795058 10628 ts_tablet_manager.cc:1403] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.295s	user 0.020s	sys 0.014s
I20260709 12:09:19.796530 10512 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.801438 10628 raft_consensus.cc:359] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.802219 10628 raft_consensus.cc:385] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.802506 10628 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.742856 10635 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:19.803647 10628 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.767335 10629 raft_consensus.cc:385] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.814839 10628 ts_tablet_manager.cc:1434] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.002s
I20260709 12:09:19.823169 10658 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:19.825832 10501 tablet_service.cc:1511] Processing CreateTablet for tablet 73ade8620f2544acbb1e92193cf221b7 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 12:09:19.809557 10553 leader_election.cc:343] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.817830 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:19.827494 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.831640 10634 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:19.832003 10553 leader_election.cc:343] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.801074 10569 tablet_service.cc:1511] Processing CreateTablet for tablet d62573c8e5a64ece95bc46d5fa18185c (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:09:19.818446 10494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d62573c8e5a64ece95bc46d5fa18185c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.815411 10629 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.840523 10427 tablet_service.cc:1511] Processing CreateTablet for tablet 73ade8620f2544acbb1e92193cf221b7 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:09:19.842015 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ade8620f2544acbb1e92193cf221b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.843408 10627 tablet_bootstrap.cc:492] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:19.858191 10512 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
W20260709 12:09:19.833226 10403 leader_election.cc:343] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.838230 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:19.861008 10634 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.863909 10629 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.868727 10577 tablet_service.cc:1511] Processing CreateTablet for tablet 73ade8620f2544acbb1e92193cf221b7 (DEFAULT_TABLE table=test-workload [id=044b427497c843d3931346454f4fb614]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:09:19.889129 10627 tablet_bootstrap.cc:654] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:19.891263 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60889.3us
W20260709 12:09:19.891173 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59200.4us
I20260709 12:09:19.894106 10635 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.894673 10635 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.898581 10635 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:19.894134 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:09:19.901782 10403 leader_election.cc:343] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.905939 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:19.902751 10635 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.909441 10635 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:19.911509 10635 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.923985 10577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ade8620f2544acbb1e92193cf221b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.924341 10628 tablet_bootstrap.cc:492] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:19.925464 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:19.926492 10586 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:19.928911 10569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d62573c8e5a64ece95bc46d5fa18185c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.930012 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:19.931003 10629 ts_tablet_manager.cc:1434] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.216s	user 0.005s	sys 0.007s
I20260709 12:09:19.936362 10501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73ade8620f2544acbb1e92193cf221b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:19.937697 10629 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:19.939884 10403 leader_election.cc:343] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.940734 10663 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.942668 10663 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.943356 10663 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.952939 10660 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.953778 10660 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:19.959825 10660 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:19.959959 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:19.960778 10436 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:19.965710 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:19.967157 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:19.969977 10552 leader_election.cc:343] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.971841 10660 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.972864 10660 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.973731 10660 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.828109 10658 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:20.004058 10629 tablet_bootstrap.cc:654] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.005453 10660 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.016970 10627 tablet_bootstrap.cc:492] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:20.017647 10627 ts_tablet_manager.cc:1403] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.180s	user 0.015s	sys 0.011s
I20260709 12:09:20.027179 10628 tablet_bootstrap.cc:654] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:20.030932 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:20.031800 10627 raft_consensus.cc:359] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.032662 10627 raft_consensus.cc:385] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.032968 10627 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.034094 10627 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.046288 10627 ts_tablet_manager.cc:1434] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.002s
I20260709 12:09:20.048348 10627 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:20.045969 10335 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:20.052634 10660 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:20.056160 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:20.056823 10436 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.058310 10627 tablet_bootstrap.cc:654] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.054579 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:20.064067 10658 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.066356 10658 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:20.067116 10552 leader_election.cc:343] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.051578 10663 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.071744 10658 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.073395 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.077428 10663 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.079581 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:20.080570 10586 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:20.081066 10336 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.083896 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:20.084297 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:09:20.086184 10403 leader_election.cc:343] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.086429 10658 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.087189 10658 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:09:20.087610 10479 leader_election.cc:343] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.088361 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:20.088970 10586 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.087718 10658 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:20.098232 10479 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:20.100337 10668 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.101193 10668 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.105594 10668 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.109313 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:20.109542 10658 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.109748 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.112247 10437 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:09:20.113925 10658 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.116823 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:20.118088 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:20.119246 10587 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:20.119755 10479 leader_election.cc:343] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.131875 10336 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:20.134730 10586 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:20.137370 10628 tablet_bootstrap.cc:492] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:20.137005 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:20.137064 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:20.138034 10628 ts_tablet_manager.cc:1403] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.214s	user 0.019s	sys 0.003s
I20260709 12:09:20.138711 10663 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.139497 10663 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:20.139781 10658 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.142653 10628 raft_consensus.cc:359] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.143322 10628 raft_consensus.cc:385] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.143567 10628 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.144282 10628 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.146787 10628 ts_tablet_manager.cc:1434] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:09:20.147272 10663 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.147851 10628 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:20.155869 10658 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.157693 10629 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:20.158272 10629 ts_tablet_manager.cc:1403] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.221s	user 0.020s	sys 0.006s
I20260709 12:09:20.159930 10627 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:20.160691 10627 ts_tablet_manager.cc:1403] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.113s	user 0.027s	sys 0.032s
I20260709 12:09:20.161729 10629 raft_consensus.cc:359] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.162545 10629 raft_consensus.cc:385] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.162842 10629 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.165460 10627 raft_consensus.cc:359] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.166659 10627 raft_consensus.cc:385] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.167223 10627 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.168471 10627 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.172170 10627 ts_tablet_manager.cc:1434] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 12:09:20.173840 10627 tablet_bootstrap.cc:492] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:20.166254 10629 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.182062 10436 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:20.183911 10627 tablet_bootstrap.cc:654] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.185441 10629 ts_tablet_manager.cc:1434] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260709 12:09:20.180733 10628 tablet_bootstrap.cc:654] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.187330 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:20.178431 10587 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:20.190152 10660 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.192415 10336 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.193718 10660 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:20.195034 10660 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.193048 10629 tablet_bootstrap.cc:492] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:20.199075 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:20.201462 10658 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.202183 10658 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:20.203572 10658 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.218768 10663 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.219457 10663 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.222654 10663 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.221692 10628 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:20.223943 10628 ts_tablet_manager.cc:1403] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.017s
I20260709 12:09:20.225405 10627 tablet_bootstrap.cc:492] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:20.226071 10627 ts_tablet_manager.cc:1403] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.015s
I20260709 12:09:20.228181 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:20.228557 10628 raft_consensus.cc:359] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.229106 10587 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:20.229370 10628 raft_consensus.cc:385] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.229714 10628 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.231040 10628 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.232154 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:20.232285 10627 raft_consensus.cc:359] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.233548 10627 raft_consensus.cc:385] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.234160 10628 ts_tablet_manager.cc:1434] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:09:20.234033 10627 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.234463 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:20.235694 10663 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.236270 10663 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.239290 10663 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.235405 10627 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:20.236760 10403 leader_election.cc:343] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.242527 10668 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.247385 10627 ts_tablet_manager.cc:1434] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.007s
I20260709 12:09:20.249141 10627 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:20.251672 10336 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.260856 10628 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:20.262629 10627 tablet_bootstrap.cc:654] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.272378 10628 tablet_bootstrap.cc:654] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.277537 10629 tablet_bootstrap.cc:654] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.278712 10663 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.282428 10663 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.284137 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:20.285210 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:20.286108 10586 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:20.289840 10403 leader_election.cc:343] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.295923 10335 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:20.329108 10629 tablet_bootstrap.cc:492] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:20.329542 10586 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:20.331984 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:20.333743 10663 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.334451 10663 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:20.335934 10663 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.347263 10660 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.348500 10634 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:20.341360 10629 ts_tablet_manager.cc:1403] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.138s	user 0.018s	sys 0.018s
I20260709 12:09:20.360695 10658 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.361518 10658 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.362502 10629 raft_consensus.cc:359] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.369648 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.365191 10629 raft_consensus.cc:385] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.371961 10629 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.373965 10628 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:20.373252 10629 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.374749 10628 ts_tablet_manager.cc:1403] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.114s	user 0.008s	sys 0.045s
I20260709 12:09:20.378073 10629 ts_tablet_manager.cc:1434] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.004s
I20260709 12:09:20.380347 10629 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:20.384086 10627 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:20.384866 10627 ts_tablet_manager.cc:1403] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.136s	user 0.024s	sys 0.039s
I20260709 12:09:20.393508 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:20.394865 10628 raft_consensus.cc:359] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.396495 10587 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:09:20.398453 10478 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:20.399780 10640 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.396123 10628 raft_consensus.cc:385] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.400538 10628 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.400593 10336 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.401790 10628 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.405529 10628 ts_tablet_manager.cc:1434] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:09:20.406885 10628 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:20.409296 10587 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.395331 10658 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.413147 10627 raft_consensus.cc:359] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.414326 10627 raft_consensus.cc:385] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.414778 10627 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.416189 10627 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.419272 10628 tablet_bootstrap.cc:654] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.419402 10627 ts_tablet_manager.cc:1434] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.001s
I20260709 12:09:20.425143 10627 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:20.438310 10627 tablet_bootstrap.cc:654] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.462702 10658 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.463587 10658 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.468993 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.470357 10437 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:09:20.471345 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:20.479355 10479 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:20.478165 10658 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.482216 10668 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.480595 10629 tablet_bootstrap.cc:654] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.480114 10640 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.499826 10628 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:20.500577 10628 ts_tablet_manager.cc:1403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.094s	user 0.027s	sys 0.039s
I20260709 12:09:20.502487 10635 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.504536 10635 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.506551 10628 raft_consensus.cc:359] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.507481 10628 raft_consensus.cc:385] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.507825 10628 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.508783 10628 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.507962 10658 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:09:20.531193 10628 ts_tablet_manager.cc:1434] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 12:09:20.532486 10628 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:20.536330 10627 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:20.537138 10627 ts_tablet_manager.cc:1403] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.112s	user 0.013s	sys 0.022s
I20260709 12:09:20.542429 10629 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:20.543181 10629 ts_tablet_manager.cc:1403] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.163s	user 0.019s	sys 0.014s
I20260709 12:09:20.551304 10629 raft_consensus.cc:359] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.553928 10629 raft_consensus.cc:385] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.554325 10629 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.555325 10629 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.557401 10628 tablet_bootstrap.cc:654] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.557049 10627 raft_consensus.cc:359] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.557765 10627 raft_consensus.cc:385] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.545605 10668 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.558142 10627 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.559708 10627 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.560623 10634 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.561407 10634 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.564420 10629 ts_tablet_manager.cc:1434] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 12:09:20.564996 10627 ts_tablet_manager.cc:1434] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.004s
I20260709 12:09:20.565874 10634 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:20.566287 10627 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:20.574307 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:20.577474 10629 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:20.578954 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:20.579205 10663 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.580011 10663 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:20.587721 10552 leader_election.cc:343] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.588764 10663 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:20.586867 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.590101 10436 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:20.596139 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:20.596338 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:20.597676 10513 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:20.598500 10634 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.599107 10634 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.599647 10634 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.600724 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:20.601969 10663 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.601766 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:20.602555 10663 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.602843 10587 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:20.603251 10663 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.604912 10629 tablet_bootstrap.cc:654] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.634897 10628 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:20.635615 10628 ts_tablet_manager.cc:1403] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.008s
I20260709 12:09:20.636686 10634 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.640832 10663 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.643730 10627 tablet_bootstrap.cc:654] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.650532 10628 raft_consensus.cc:359] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.649199 10660 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.651278 10628 raft_consensus.cc:385] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.651913 10628 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.651805 10660 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.653074 10628 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.655966 10628 ts_tablet_manager.cc:1434] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 12:09:20.659224 10628 tablet_bootstrap.cc:492] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:20.663956 10629 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:20.664707 10629 ts_tablet_manager.cc:1403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.096s	user 0.015s	sys 0.010s
I20260709 12:09:20.658529 10660 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:20.667112 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:20.669016 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:20.671290 10552 leader_election.cc:343] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.675570 10640 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.683967 10634 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:20.682602 10629 raft_consensus.cc:359] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.685420 10629 raft_consensus.cc:385] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.685812 10629 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.686856 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:20.686658 10629 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.701114 10629 ts_tablet_manager.cc:1434] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.003s
I20260709 12:09:20.702327 10629 tablet_bootstrap.cc:492] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:20.698752 10436 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:20.710168 10663 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:20.706874 10552 leader_election.cc:343] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.713945 10437 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.724748 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:20.725231 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:20.725773 10438 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.732141 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:20.733193 10512 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.737555 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:20.738688 10587 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.752589 10703 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.758875 10704 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:09:20.763825 10627 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:20.764050 10703 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.765815 10705 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.766552 10705 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.767114 10705 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.768215 10627 ts_tablet_manager.cc:1403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.202s	user 0.013s	sys 0.026s
I20260709 12:09:20.807508 10640 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:09:20.819834 10628 tablet_bootstrap.cc:654] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.790632 10512 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:20.806224 10438 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:20.822691 10627 raft_consensus.cc:359] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.823801 10627 raft_consensus.cc:385] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.823634 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:20.824247 10627 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.825732 10660 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.793128 10511 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.797998 10634 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.828503 10634 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.830693 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:20.833532 10703 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.834259 10703 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:20.794093 10704 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260709 12:09:20.835793 10703 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.837775 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:20.844594 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.846844 10704 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.851157 10704 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.851615 10663 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.854818 10629 tablet_bootstrap.cc:654] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.854031 10627 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.853288 10634 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:09:20.849036 10552 leader_election.cc:343] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.895828 10627 ts_tablet_manager.cc:1434] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.078s	user 0.011s	sys 0.002s
I20260709 12:09:20.897363 10627 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:20.893734 10587 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
W20260709 12:09:20.898756 10553 leader_election.cc:343] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.892071 10660 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:09:20.894142 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 53063.7us
I20260709 12:09:20.937445 10660 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.945731 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:20.921463 10704 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:20.951885 10634 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.969291 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:20.974799 10552 leader_election.cc:343] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.982591 10629 tablet_bootstrap.cc:492] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
W20260709 12:09:20.963418 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116189us
I20260709 12:09:20.983352 10629 ts_tablet_manager.cc:1403] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.281s	user 0.026s	sys 0.007s
I20260709 12:09:20.984834 10668 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.985994 10668 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.988365 10628 tablet_bootstrap.cc:492] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:20.988238 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:20.989102 10628 ts_tablet_manager.cc:1403] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.332s	user 0.033s	sys 0.005s
I20260709 12:09:20.988445 10705 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.992316 10705 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:20.996898 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:20.997524 10628 raft_consensus.cc:359] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:20.998020 10513 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.998432 10628 raft_consensus.cc:385] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.998826 10628 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
W20260709 12:09:21.013175 10553 leader_election.cc:343] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.012956 10628 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.014083 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.015940 10705 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.015995 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:21.016997 10436 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.019907 10668 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.024696 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.014377 10336 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:21.028978 10628 ts_tablet_manager.cc:1434] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.039s	user 0.011s	sys 0.002s
I20260709 12:09:21.035382 10628 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:21.043771 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:21.043874 10703 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:21.052764 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:21.054719 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.056715 10668 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:21.083180 10627 tablet_bootstrap.cc:654] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.084363 10629 raft_consensus.cc:359] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:21.085047 10629 raft_consensus.cc:385] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.085304 10629 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.085927 10629 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:21.089684 10629 ts_tablet_manager.cc:1434] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.105s	user 0.005s	sys 0.003s
I20260709 12:09:21.090914 10629 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:21.100658 10437 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.112896 10436 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:21.117906 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:21.119508 10634 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.120306 10634 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:21.121809 10634 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.125768 10628 tablet_bootstrap.cc:654] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.127446 10513 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:21.141372 10335 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:21.141350 10629 tablet_bootstrap.cc:654] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.193742 10628 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:21.194409 10628 ts_tablet_manager.cc:1403] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.159s	user 0.030s	sys 0.052s
I20260709 12:09:21.201828 10628 raft_consensus.cc:359] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.203075 10628 raft_consensus.cc:385] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.203534 10628 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.204661 10628 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.209359 10627 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:21.209908 10627 ts_tablet_manager.cc:1403] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.313s	user 0.017s	sys 0.016s
I20260709 12:09:21.211308 10628 ts_tablet_manager.cc:1434] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:09:21.212538 10628 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:21.219321 10627 raft_consensus.cc:359] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.225099 10627 raft_consensus.cc:385] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.225596 10627 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.226763 10629 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:21.226691 10627 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.230190 10627 ts_tablet_manager.cc:1434] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.001s
I20260709 12:09:21.228138 10629 ts_tablet_manager.cc:1403] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.137s	user 0.010s	sys 0.029s
I20260709 12:09:21.232170 10628 tablet_bootstrap.cc:654] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.236690 10335 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:21.241448 10627 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:21.246527 10629 raft_consensus.cc:359] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.247560 10629 raft_consensus.cc:385] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.247946 10629 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.249051 10629 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.253248 10629 ts_tablet_manager.cc:1434] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.002s
I20260709 12:09:21.255292 10629 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:21.256383 10627 tablet_bootstrap.cc:654] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.261477 10703 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.263696 10628 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:21.264406 10628 ts_tablet_manager.cc:1403] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.052s	user 0.006s	sys 0.015s
I20260709 12:09:21.269908 10628 raft_consensus.cc:359] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.271605 10628 raft_consensus.cc:385] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.271914 10628 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.275174 10628 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.286039 10628 ts_tablet_manager.cc:1434] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:09:21.289227 10628 tablet_bootstrap.cc:492] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:21.303313 10658 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.304311 10658 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.309304 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:21.310943 10437 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:09:21.315913 10658 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.321477 10627 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:21.322393 10627 ts_tablet_manager.cc:1403] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.081s	user 0.028s	sys 0.036s
I20260709 12:09:21.324012 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.326902 10627 raft_consensus.cc:359] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.327832 10627 raft_consensus.cc:385] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.328230 10627 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.329149 10627 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.330477 10629 tablet_bootstrap.cc:654] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.330102 10478 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.332406 10627 ts_tablet_manager.cc:1434] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.005s
I20260709 12:09:21.333636 10627 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:21.331070 10703 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.336438 10658 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.338183 10628 tablet_bootstrap.cc:654] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.369029 10627 tablet_bootstrap.cc:654] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.389927 10629 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:21.390971 10629 ts_tablet_manager.cc:1403] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.136s	user 0.030s	sys 0.013s
I20260709 12:09:21.397913 10658 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.408294 10703 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.409085 10703 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.414357 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:21.423182 10629 raft_consensus.cc:359] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.424500 10629 raft_consensus.cc:385] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.424921 10629 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.425910 10629 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.428819 10703 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.433665 10629 ts_tablet_manager.cc:1434] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.003s
I20260709 12:09:21.433387 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.436076 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:21.436076 10629 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:21.438593 10703 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.442881 10627 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:21.443542 10627 ts_tablet_manager.cc:1403] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.110s	user 0.021s	sys 0.026s
I20260709 12:09:21.447189 10634 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.447877 10634 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.453626 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:21.454622 10634 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:21.454182 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:21.456045 10553 leader_election.cc:343] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:21.457712 10552 leader_election.cc:343] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.458425 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.461603 10634 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.450184 10627 raft_consensus.cc:359] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.465476 10627 raft_consensus.cc:385] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.465852 10627 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.467424 10628 tablet_bootstrap.cc:492] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:21.468091 10627 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.474438 10628 ts_tablet_manager.cc:1403] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.186s	user 0.013s	sys 0.032s
I20260709 12:09:21.476526 10629 tablet_bootstrap.cc:654] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.477691 10628 raft_consensus.cc:359] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.478363 10628 raft_consensus.cc:385] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.478688 10628 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.479502 10705 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.479588 10628 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.480487 10627 ts_tablet_manager.cc:1434] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:09:21.480194 10705 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.481786 10627 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:21.486863 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:21.489192 10660 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:21.497469 10553 leader_election.cc:343] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.498174 10628 ts_tablet_manager.cc:1434] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.004s
I20260709 12:09:21.500582 10627 tablet_bootstrap.cc:654] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.503212 10628 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:21.509950 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:21.501981 10629 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
W20260709 12:09:21.512991 10552 leader_election.cc:343] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.513924 10629 ts_tablet_manager.cc:1403] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.022s
I20260709 12:09:21.513809 10552 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.516091 10660 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.518371 10703 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.519225 10703 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.521380 10628 tablet_bootstrap.cc:654] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.522487 10703 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.522487 10629 raft_consensus.cc:359] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.524510 10629 raft_consensus.cc:385] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.524840 10629 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.526085 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.525941 10629 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.526196 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:21.526978 10584 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:21.529119 10629 ts_tablet_manager.cc:1434] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.008s
I20260709 12:09:21.529203 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:21.530500 10629 tablet_bootstrap.cc:492] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:21.532053 10403 leader_election.cc:343] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.530670 10705 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:21.532815 10703 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.537473 10627 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:21.538029 10703 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.539274 10703 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.538221 10627 ts_tablet_manager.cc:1403] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.020s
I20260709 12:09:21.550595 10627 raft_consensus.cc:359] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.551481 10627 raft_consensus.cc:385] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.551818 10627 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.552650 10627 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.553180 10635 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.553995 10635 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.557845 10635 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.560923 10627 ts_tablet_manager.cc:1434] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:09:21.566840 10628 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:21.571967 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.573328 10584 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:21.573843 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:21.574666 10629 tablet_bootstrap.cc:654] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.575410 10628 ts_tablet_manager.cc:1403] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.073s	user 0.009s	sys 0.025s
W20260709 12:09:21.576347 10403 leader_election.cc:343] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.576684 10627 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:21.583993 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:21.585377 10635 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.585916 10635 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.580881 10628 raft_consensus.cc:359] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.581104 10705 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.586421 10635 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.590029 10628 raft_consensus.cc:385] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.591470 10628 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.592564 10628 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.593652 10703 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.595873 10628 ts_tablet_manager.cc:1434] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:09:21.597167 10628 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:21.598951 10627 tablet_bootstrap.cc:654] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.605069 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:21.606184 10584 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.607172 10635 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.606063 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:21.613389 10668 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.614166 10668 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.617067 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:21.617399 10403 leader_election.cc:343] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.608394 10703 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.563374 10660 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.080s
I20260709 12:09:21.652848 10628 tablet_bootstrap.cc:654] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.621410 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:21.616508 10635 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:21.658759 10403 leader_election.cc:343] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.645677 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.648921 10438 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.660202 10587 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.663872 10586 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:21.668416 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:21.669606 10658 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.670058 10658 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.670454 10658 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.662681 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:21.672525 10479 leader_election.cc:343] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.622555 10668 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.687759 10660 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.691473 10584 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:21.696460 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:21.704131 10703 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.704169 10627 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:21.705329 10627 ts_tablet_manager.cc:1403] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.129s	user 0.017s	sys 0.017s
I20260709 12:09:21.708402 10703 raft_consensus.cc:697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:21.712391 10703 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.717077 10658 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.718026 10629 tablet_bootstrap.cc:492] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:21.724675 10629 ts_tablet_manager.cc:1403] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.195s	user 0.018s	sys 0.011s
I20260709 12:09:21.724103 10587 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:21.720412 10627 raft_consensus.cc:359] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.726658 10627 raft_consensus.cc:385] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.727401 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:21.730244 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:21.735920 10627 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.737121 10627 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.738719 10629 raft_consensus.cc:359] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.739589 10629 raft_consensus.cc:385] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.739429 10658 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.740079 10629 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.741544 10629 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.732959 10635 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:09:21.750499 10479 leader_election.cc:343] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.752727 10627 ts_tablet_manager.cc:1434] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.004s
I20260709 12:09:21.753155 10658 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.754029 10627 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:21.753826 10658 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.757139 10658 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.758906 10629 ts_tablet_manager.cc:1434] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.001s
W20260709 12:09:21.743757 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 65344.5us
I20260709 12:09:21.760161 10629 tablet_bootstrap.cc:492] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:21.764271 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:21.764618 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:21.765216 10587 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.803773 10743 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.804473 10743 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.807967 10628 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:21.809165 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:21.811513 10479 leader_election.cc:343] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.827524 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.828861 10586 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:21.843178 10658 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.844118 10658 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.845182 10743 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.860424 10634 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.861603 10628 ts_tablet_manager.cc:1403] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.265s	user 0.017s	sys 0.023s
I20260709 12:09:21.818028 10587 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:21.861397 10634 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.869519 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.874660 10747 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.875140 10747 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.876421 10747 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.876628 10634 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:21.877066 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:21.868642 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:09:21.877851 10479 leader_election.cc:343] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:21.878753 10552 leader_election.cc:343] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.879298 10658 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.880168 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.881127 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:21.882606 10479 leader_election.cc:343] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.883116 10668 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.883582 10668 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:21.884467 10668 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.888130 10635 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:21.893944 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:21.902005 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:21.903012 10587 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
W20260709 12:09:21.908892 10553 leader_election.cc:343] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.909641 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.911275 10634 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.904992 10478 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.914383 10627 tablet_bootstrap.cc:654] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.916266 10742 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.935441 10747 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:21.930330 10478 leader_election.cc:343] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.937258 10478 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.938884 10747 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.931667 10628 raft_consensus.cc:359] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.940526 10742 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.940716 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:21.949630 10742 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.941607 10628 raft_consensus.cc:385] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.950417 10628 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
W20260709 12:09:21.951061 10479 leader_election.cc:343] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.951547 10628 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.954077 10628 ts_tablet_manager.cc:1434] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.000s
I20260709 12:09:21.943701 10629 tablet_bootstrap.cc:654] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.942021 10635 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.947983 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:21.955520 10584 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.949061 10747 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.967254 10742 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:21.971261 10742 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:21.971167 10584 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:21.972347 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:09:21.974720 10479 leader_election.cc:343] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.977097 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:21.981019 10628 tablet_bootstrap.cc:492] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:21.985594 10742 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.990095 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:21.994812 10584 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.994269 10742 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:22.002766 10627 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:22.003031 10629 tablet_bootstrap.cc:492] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:22.003350 10627 ts_tablet_manager.cc:1403] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.250s	user 0.009s	sys 0.025s
I20260709 12:09:22.003651 10629 ts_tablet_manager.cc:1403] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.244s	user 0.020s	sys 0.012s
I20260709 12:09:22.009150 10742 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.018240 10628 tablet_bootstrap.cc:654] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.023766 10629 raft_consensus.cc:359] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.012699 10335 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.024801 10629 raft_consensus.cc:385] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.025161 10629 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.026726 10335 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.027813 10629 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.032020 10629 ts_tablet_manager.cc:1434] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:09:22.033950 10627 raft_consensus.cc:359] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.034426 10627 raft_consensus.cc:385] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.034593 10627 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.035184 10627 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.036888 10627 ts_tablet_manager.cc:1434] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.000s
I20260709 12:09:22.037453 10635 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.037640 10629 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:22.037822 10627 tablet_bootstrap.cc:492] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:22.037974 10635 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.040560 10584 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:22.040110 10336 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.042181 10478 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.042793 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:22.045249 10403 leader_election.cc:343] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.046270 10742 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.046890 10635 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:22.049373 10742 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:22.049619 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:22.050501 10742 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.065181 10628 tablet_bootstrap.cc:492] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:22.065706 10628 ts_tablet_manager.cc:1403] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.011s
W20260709 12:09:22.063050 10403 leader_election.cc:343] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.067346 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:22.068518 10635 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.074326 10628 raft_consensus.cc:359] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.075193 10628 raft_consensus.cc:385] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.075500 10628 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.076403 10628 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.079101 10628 ts_tablet_manager.cc:1434] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:09:22.081518 10628 tablet_bootstrap.cc:492] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:22.089530 10629 tablet_bootstrap.cc:654] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.109786 10627 tablet_bootstrap.cc:654] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.110199 10628 tablet_bootstrap.cc:654] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.126067 10635 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.127107 10635 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.131788 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.132530 10635 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:22.133635 10403 leader_election.cc:343] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.135761 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:22.136708 10584 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
W20260709 12:09:22.142334 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:22.143653 10627 tablet_bootstrap.cc:492] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:22.138757 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:22.144227 10627 ts_tablet_manager.cc:1403] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.107s	user 0.020s	sys 0.014s
I20260709 12:09:22.146312 10635 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.146900 10635 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.147461 10635 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.160446 10663 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.167358 10629 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:22.168095 10629 ts_tablet_manager.cc:1403] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.131s	user 0.033s	sys 0.030s
I20260709 12:09:22.179632 10627 raft_consensus.cc:359] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.180665 10627 raft_consensus.cc:385] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.180938 10627 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.182085 10627 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.188575 10629 raft_consensus.cc:359] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.189498 10629 raft_consensus.cc:385] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.189805 10629 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.190699 10629 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.192072 10627 ts_tablet_manager.cc:1434] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:09:22.193282 10627 tablet_bootstrap.cc:492] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:22.193578 10629 ts_tablet_manager.cc:1434] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:09:22.194666 10629 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:22.196507 10628 tablet_bootstrap.cc:492] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:22.196874 10628 ts_tablet_manager.cc:1403] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.116s	user 0.018s	sys 0.010s
I20260709 12:09:22.188604 10635 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.213200 10635 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:22.215698 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:22.216198 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:22.220499 10511 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.220774 10587 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.222630 10628 raft_consensus.cc:359] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.223480 10628 raft_consensus.cc:385] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.223687 10628 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.234642 10628 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.236370 10511 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:22.236428 10587 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:22.237514 10336 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.241118 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:22.242475 10628 ts_tablet_manager.cc:1434] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.045s	user 0.011s	sys 0.000s
I20260709 12:09:22.243257 10635 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.243995 10628 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:22.243995 10635 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:22.245579 10336 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.254331 10627 tablet_bootstrap.cc:654] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.257061 10635 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.268033 10629 tablet_bootstrap.cc:654] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.288256 10742 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.289119 10742 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.294294 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:22.295773 10403 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:09:22.296731 10479 leader_election.cc:343] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.301565 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:22.305661 10628 tablet_bootstrap.cc:654] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.299563 10742 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:22.304021 10584 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:22.313140 10627 tablet_bootstrap.cc:492] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:22.313716 10627 ts_tablet_manager.cc:1403] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.121s	user 0.018s	sys 0.002s
W20260709 12:09:22.315728 10479 consensus_peers.cc:597] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:22.318065 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.322695 10629 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:22.323355 10629 ts_tablet_manager.cc:1403] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.129s	user 0.019s	sys 0.008s
I20260709 12:09:22.324507 10668 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.326049 10668 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.326658 10668 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.335923 10627 raft_consensus.cc:359] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.339944 10627 raft_consensus.cc:385] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.340301 10627 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.341473 10629 raft_consensus.cc:359] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.342144 10629 raft_consensus.cc:385] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.342340 10629 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.345959 10628 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:22.346421 10628 ts_tablet_manager.cc:1403] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.103s	user 0.008s	sys 0.010s
I20260709 12:09:22.347424 10629 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.350067 10663 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.350852 10663 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.351398 10627 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.352275 10628 raft_consensus.cc:359] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.354645 10629 ts_tablet_manager.cc:1434] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.006s
I20260709 12:09:22.355325 10668 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.356879 10629 tablet_bootstrap.cc:492] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:22.357287 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:22.355801 10627 ts_tablet_manager.cc:1434] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.009s
I20260709 12:09:22.358664 10627 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:22.359373 10587 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:09:22.360728 10663 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:22.356914 10334 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.352998 10628 raft_consensus.cc:385] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.362905 10628 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.364032 10628 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.363903 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.367570 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:22.368798 10663 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.371626 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:22.372601 10438 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.380050 10705 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.381600 10628 ts_tablet_manager.cc:1434] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 12:09:22.381346 10705 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.381982 10668 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:22.384107 10628 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:22.392624 10635 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.069s
I20260709 12:09:22.463653 10629 tablet_bootstrap.cc:654] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.395126 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:22.470999 10627 tablet_bootstrap.cc:654] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.432458 10634 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.473482 10587 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.474303 10634 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.418053 10660 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.478102 10668 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.481869 10663 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.485327 10438 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:22.486393 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:22.486673 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.487641 10511 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:22.489748 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:22.484802 10705 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:09:22.491688 10553 leader_election.cc:343] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.492120 10705 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.492712 10705 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.493197 10705 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.496448 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:22.478307 10660 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.504133 10628 tablet_bootstrap.cc:654] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:22.502797 10403 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:22.508035 10668 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.510099 10660 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.510800 10668 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:22.512241 10668 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:22.517297 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:22.536384 10705 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:22.537577 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47997us
I20260709 12:09:22.540287 10587 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:22.557145 10627 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:22.557813 10627 ts_tablet_manager.cc:1403] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.199s	user 0.020s	sys 0.026s
I20260709 12:09:22.563577 10628 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:22.564297 10628 ts_tablet_manager.cc:1403] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.180s	user 0.053s	sys 0.044s
I20260709 12:09:22.575423 10742 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.578683 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:22.580554 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:22.581550 10437 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:22.582729 10628 raft_consensus.cc:359] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:22.584206 10553 leader_election.cc:343] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.586887 10705 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.588814 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.589774 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:22.594214 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:22.594919 10513 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.596897 10774 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.597229 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:22.597466 10774 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.597944 10634 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.598027 10774 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.592162 10628 raft_consensus.cc:385] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.600041 10628 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.606683 10742 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:09:22.610718 10628 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:22.608585 10552 leader_election.cc:343] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.627808 10628 ts_tablet_manager.cc:1434] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.003s
W20260709 12:09:22.623520 10553 leader_election.cc:343] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.651788 10747 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.649744 10336 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.634536 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.658010 10511 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:22.662914 10627 raft_consensus.cc:359] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.664323 10627 raft_consensus.cc:385] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.667877 10627 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.669284 10552 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.676707 10627 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.679360 10628 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:22.679235 10773 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.680377 10773 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.686782 10773 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.697038 10513 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:22.697191 10774 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.685029 10705 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.700964 10627 ts_tablet_manager.cc:1434] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.143s	user 0.003s	sys 0.009s
I20260709 12:09:22.701903 10627 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:22.703500 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.712116 10629 tablet_bootstrap.cc:492] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:22.712822 10629 ts_tablet_manager.cc:1403] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.356s	user 0.017s	sys 0.038s
I20260709 12:09:22.717338 10629 raft_consensus.cc:359] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.701022 10705 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.718303 10629 raft_consensus.cc:385] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.721413 10705 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.746318 10629 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.747182 10629 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.750346 10629 ts_tablet_manager.cc:1434] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 12:09:22.728159 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:22.724380 10552 leader_election.cc:343] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.730569 10742 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.760216 10742 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.761142 10629 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:22.769869 10553 leader_election.cc:343] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.770748 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.730777 10774 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.735018 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:22.771993 10634 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.773248 10436 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.779572 10634 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.780483 10634 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.782820 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:22.784983 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48088.1us
I20260709 12:09:22.786146 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.787478 10774 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.788241 10774 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:22.789578 10774 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.791838 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:22.796288 10742 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:22.800237 10663 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.801442 10634 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.739656 10627 tablet_bootstrap.cc:654] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.804942 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:22.807044 10553 leader_election.cc:343] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.808071 10629 tablet_bootstrap.cc:654] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.744035 10628 tablet_bootstrap.cc:654] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:22.814270 10552 leader_election.cc:343] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.833326 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:22.835412 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:22.842780 10511 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.847793 10705 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.854046 10478 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.861315 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:09:22.862941 10553 leader_election.cc:343] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.857220 10705 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:22.861716 10333 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:22.873708 10750 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:22.890250 10552 leader_election.cc:343] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.891033 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.892099 10705 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.893723 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:22.894521 10513 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.896411 10628 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:22.896965 10628 ts_tablet_manager.cc:1403] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.218s	user 0.015s	sys 0.015s
I20260709 12:09:22.904821 10513 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:22.907075 10552 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:22.846855 10436 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:22.908789 10705 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.909515 10705 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:22.910514 10627 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:22.910821 10705 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.911336 10628 raft_consensus.cc:359] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.912117 10628 raft_consensus.cc:385] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.912431 10628 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.913128 10627 ts_tablet_manager.cc:1403] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.211s	user 0.017s	sys 0.021s
I20260709 12:09:22.915822 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:22.917959 10629 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:22.919184 10629 ts_tablet_manager.cc:1403] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.158s	user 0.014s	sys 0.018s
I20260709 12:09:22.917927 10774 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.914951 10628 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.931792 10774 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:22.940961 10774 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.934182 10628 ts_tablet_manager.cc:1434] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 12:09:22.945878 10788 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.948843 10628 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:22.928040 10629 raft_consensus.cc:359] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.954252 10629 raft_consensus.cc:385] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.960119 10629 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.954174 10788 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:09:22.960881 10629 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.959682 10627 raft_consensus.cc:359] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.964100 10627 raft_consensus.cc:385] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.964394 10627 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.965287 10627 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:22.974974 10629 ts_tablet_manager.cc:1434] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.055s	user 0.010s	sys 0.001s
I20260709 12:09:22.976675 10629 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:22.978669 10788 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.984735 10627 ts_tablet_manager.cc:1434] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.004s
I20260709 12:09:22.986256 10627 tablet_bootstrap.cc:492] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:22.996168 10628 tablet_bootstrap.cc:654] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.011176 10629 tablet_bootstrap.cc:654] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.055341 10627 tablet_bootstrap.cc:654] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.061276 10628 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:23.061885 10628 ts_tablet_manager.cc:1403] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.113s	user 0.021s	sys 0.000s
I20260709 12:09:23.072182 10628 raft_consensus.cc:359] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.073141 10628 raft_consensus.cc:385] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.083865 10629 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:23.085613 10628 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.085872 10629 ts_tablet_manager.cc:1403] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.109s	user 0.038s	sys 0.019s
I20260709 12:09:23.090147 10629 raft_consensus.cc:359] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.086757 10628 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.090880 10629 raft_consensus.cc:385] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.091696 10629 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.093101 10629 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.095597 10629 ts_tablet_manager.cc:1434] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:09:23.096786 10629 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:23.107543 10629 tablet_bootstrap.cc:654] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.110244 10627 tablet_bootstrap.cc:492] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:23.110944 10627 ts_tablet_manager.cc:1403] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.125s	user 0.015s	sys 0.003s
I20260709 12:09:23.093703 10335 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.114399 10335 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.126019 10628 ts_tablet_manager.cc:1434] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.064s	user 0.012s	sys 0.000s
I20260709 12:09:23.127131 10628 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:23.142544 10628 tablet_bootstrap.cc:654] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.143884 10627 raft_consensus.cc:359] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.145044 10627 raft_consensus.cc:385] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.145402 10627 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.146430 10627 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.167505 10627 ts_tablet_manager.cc:1434] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.000s
I20260709 12:09:23.171566 10629 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:23.172168 10629 ts_tablet_manager.cc:1403] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.076s	user 0.031s	sys 0.042s
I20260709 12:09:23.177884 10629 raft_consensus.cc:359] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.178673 10629 raft_consensus.cc:385] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.179033 10629 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.180719 10627 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:23.188100 10629 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.192054 10629 ts_tablet_manager.cc:1434] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.003s
I20260709 12:09:23.193392 10629 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:23.208024 10627 tablet_bootstrap.cc:654] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.209652 10629 tablet_bootstrap.cc:654] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.222097 10628 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:23.222855 10628 ts_tablet_manager.cc:1403] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.015s
W20260709 12:09:23.224843 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:23.228289 10774 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.230839 10627 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:23.231449 10627 ts_tablet_manager.cc:1403] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.004s
I20260709 12:09:23.241474 10627 raft_consensus.cc:359] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.242591 10627 raft_consensus.cc:385] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.242973 10627 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.245044 10627 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.245935 10663 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.246411 10635 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.247143 10663 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.247224 10635 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.250804 10627 ts_tablet_manager.cc:1434] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.004s
I20260709 12:09:23.250757 10635 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.250975 10663 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.252255 10627 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:23.254189 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.252985 10628 raft_consensus.cc:359] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.255860 10628 raft_consensus.cc:385] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.256182 10628 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.255052 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.256331 10774 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.257302 10628 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.258051 10774 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.259022 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.261279 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:23.261682 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.262712 10511 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:09:23.263535 10635 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.263732 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.264355 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:23.264945 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.263724 10774 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.265913 10510 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:23.266712 10663 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:23.266633 10553 leader_election.cc:343] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.267877 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.269824 10774 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.270412 10774 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.271040 10774 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.273448 10628 ts_tablet_manager.cc:1434] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.003s
I20260709 12:09:23.274535 10628 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:23.276441 10629 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:23.277523 10629 ts_tablet_manager.cc:1403] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.084s	user 0.016s	sys 0.025s
I20260709 12:09:23.282722 10785 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.283483 10785 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.300254 10436 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.300146 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:23.302599 10479 leader_election.cc:343] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.299425 10785 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.307991 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.309208 10587 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:23.319736 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.322105 10742 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.322683 10742 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.323267 10742 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.329504 10437 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.331772 10635 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.332471 10635 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.337936 10627 tablet_bootstrap.cc:654] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.337811 10635 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.344422 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.344748 10634 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.346081 10634 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.367718 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.370131 10705 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.376061 10634 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.377010 10634 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.378522 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:23.381364 10552 leader_election.cc:343] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.381850 10663 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.383590 10628 tablet_bootstrap.cc:654] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.388320 10629 raft_consensus.cc:359] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.389106 10629 raft_consensus.cc:385] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.389462 10629 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.389951 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.390156 10629 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.396930 10436 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:23.408792 10629 ts_tablet_manager.cc:1434] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.131s	user 0.005s	sys 0.012s
I20260709 12:09:23.410351 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:23.411566 10705 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.420993 10629 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:23.388828 10774 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.421927 10663 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.437994 10742 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.429765 10773 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.443568 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:23.444367 10587 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:23.454655 10552 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:23.442888 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:23.441232 10773 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.441866 10742 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:23.506101 10479 leader_election.cc:343] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.507141 10703 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:23.508085 10774 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.508571 10703 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.510515 10438 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.510622 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:23.511606 10513 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.515640 10703 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.543722 10627 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:23.547545 10587 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:23.571679 10627 ts_tablet_manager.cc:1403] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.320s	user 0.028s	sys 0.029s
I20260709 12:09:23.573560 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.575655 10785 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.575555 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:23.576324 10785 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:23.577248 10774 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.577812 10774 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.577329 10785 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.578301 10774 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.575289 10705 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:09:23.582250 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.583019 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.586241 10629 tablet_bootstrap.cc:654] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.587844 10628 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:23.588311 10628 ts_tablet_manager.cc:1403] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.314s	user 0.021s	sys 0.035s
I20260709 12:09:23.589697 10742 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.590286 10742 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.590662 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:23.592876 10552 leader_election.cc:343] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.595150 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.595533 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.596011 10513 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
W20260709 12:09:23.597675 10479 leader_election.cc:343] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.598155 10742 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.587589 10773 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.601620 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:23.605195 10773 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.606503 10773 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:23.608191 10628 raft_consensus.cc:359] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.608964 10628 raft_consensus.cc:385] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.609233 10628 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
W20260709 12:09:23.609313 10478 leader_election.cc:343] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.610209 10628 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.610488 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.611496 10627 raft_consensus.cc:359] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.612249 10627 raft_consensus.cc:385] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.612583 10627 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.612775 10742 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.613431 10627 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.613456 10628 ts_tablet_manager.cc:1434] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:09:23.616214 10627 ts_tablet_manager.cc:1434] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.002s
I20260709 12:09:23.616811 10628 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:23.617275 10627 tablet_bootstrap.cc:492] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:23.623250 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:23.609194 10773 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.626031 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.631023 10628 tablet_bootstrap.cc:654] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.629726 10336 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:23.633940 10629 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:23.627255 10774 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.634488 10629 ts_tablet_manager.cc:1403] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.214s	user 0.019s	sys 0.005s
I20260709 12:09:23.647388 10627 tablet_bootstrap.cc:654] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:23.649746 10553 leader_election.cc:343] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:23.651322 10553 leader_election.cc:343] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.651778 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.651733 10403 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:23.653649 10703 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:23.654068 10809 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.663314 10774 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.666842 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:23.667929 10629 raft_consensus.cc:359] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.668689 10629 raft_consensus.cc:385] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.668974 10629 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
W20260709 12:09:23.669178 10552 leader_election.cc:343] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.669896 10629 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.673568 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:23.674299 10438 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.675520 10628 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:23.676000 10628 ts_tablet_manager.cc:1403] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.004s
I20260709 12:09:23.678704 10628 raft_consensus.cc:359] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.679371 10628 raft_consensus.cc:385] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.679649 10628 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.685902 10438 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:23.680253 10628 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.688211 10629 ts_tablet_manager.cc:1434] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 12:09:23.688961 10628 ts_tablet_manager.cc:1434] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:09:23.689308 10629 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:23.690233 10628 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:23.691448 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:23.696264 10705 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.698973 10705 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:23.702391 10627 tablet_bootstrap.cc:492] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:23.701879 10705 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.703015 10627 ts_tablet_manager.cc:1403] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.007s
I20260709 12:09:23.707526 10627 raft_consensus.cc:359] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.708137 10627 raft_consensus.cc:385] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.708388 10627 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.711009 10627 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.714577 10334 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.719957 10627 ts_tablet_manager.cc:1434] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:09:23.721078 10627 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:23.729321 10774 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.730062 10774 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.733271 10774 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.734869 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.736256 10513 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:23.737427 10629 tablet_bootstrap.cc:654] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.738627 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:23.740301 10773 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.740899 10773 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.741493 10773 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.742830 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:23.745344 10553 leader_election.cc:343] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.750661 10628 tablet_bootstrap.cc:654] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.759060 10747 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.760092 10747 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.767711 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.773626 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.775619 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.776932 10742 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.778923 10747 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.788304 10773 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.789433 10628 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:23.790005 10628 ts_tablet_manager.cc:1403] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.100s	user 0.015s	sys 0.002s
I20260709 12:09:23.790419 10629 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:23.791015 10629 ts_tablet_manager.cc:1403] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.102s	user 0.019s	sys 0.032s
I20260709 12:09:23.793342 10773 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:23.791100 10627 tablet_bootstrap.cc:654] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.794159 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:23.795248 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:23.796156 10553 leader_election.cc:343] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.796234 10513 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.803808 10629 raft_consensus.cc:359] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.805706 10629 raft_consensus.cc:385] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.806030 10629 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.806689 10628 raft_consensus.cc:359] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.807718 10628 raft_consensus.cc:385] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.808476 10628 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.809361 10628 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.806828 10629 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.812187 10628 ts_tablet_manager.cc:1434] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.004s
I20260709 12:09:23.813544 10628 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:23.822467 10628 tablet_bootstrap.cc:654] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.822968 10334 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.828146 10629 ts_tablet_manager.cc:1434] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.037s	user 0.011s	sys 0.000s
I20260709 12:09:23.829321 10629 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:23.830859 10627 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:23.831434 10627 ts_tablet_manager.cc:1403] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.111s	user 0.013s	sys 0.009s
I20260709 12:09:23.835234 10627 raft_consensus.cc:359] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.836576 10627 raft_consensus.cc:385] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.837270 10627 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.838490 10627 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.841566 10629 tablet_bootstrap.cc:654] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.841715 10627 ts_tablet_manager.cc:1434] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:09:23.849539 10627 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:23.858095 10513 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:23.860445 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.862686 10773 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.863406 10773 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:23.864737 10773 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.869756 10705 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.870586 10705 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.877956 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.878695 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:23.880434 10553 leader_election.cc:343] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.881279 10705 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:09:23.883358 10552 leader_election.cc:343] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.884102 10552 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:23.885914 10705 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.898270 10628 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:23.899016 10628 ts_tablet_manager.cc:1403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.013s
I20260709 12:09:23.901690 10627 tablet_bootstrap.cc:654] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.901918 10334 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.903519 10628 raft_consensus.cc:359] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.904420 10628 raft_consensus.cc:385] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.904752 10629 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:23.904763 10628 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.905370 10629 ts_tablet_manager.cc:1403] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.076s	user 0.020s	sys 0.047s
I20260709 12:09:23.905856 10628 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.921168 10628 ts_tablet_manager.cc:1434] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 12:09:23.923214 10628 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:23.921190 10629 raft_consensus.cc:359] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.925889 10629 raft_consensus.cc:385] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.926213 10629 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.926951 10629 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.928728 10629 ts_tablet_manager.cc:1434] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.004s
I20260709 12:09:23.935030 10629 tablet_bootstrap.cc:492] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:23.941622 10627 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:23.935145 10785 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.942188 10627 ts_tablet_manager.cc:1403] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.093s	user 0.014s	sys 0.001s
I20260709 12:09:23.945559 10629 tablet_bootstrap.cc:654] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.945739 10627 raft_consensus.cc:359] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.946501 10627 raft_consensus.cc:385] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.946805 10627 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.942524 10785 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.947690 10627 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.949978 10627 ts_tablet_manager.cc:1434] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:09:23.949780 10703 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.950955 10627 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:23.951319 10703 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.959383 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.960175 10587 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:23.961674 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:23.962224 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:23.962980 10511 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:23.964341 10785 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.964556 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:23.965152 10703 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.965433 10438 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:23.965929 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:23.966885 10584 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:23.971490 10750 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.973829 10750 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:23.982178 10750 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:23.967283 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:23.986477 10747 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.986948 10747 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.987391 10747 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.993155 10513 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.975507 10703 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.000365 10703 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.000998 10703 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.031976 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:24.033093 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.034543 10511 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:09:24.039966 10747 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.042724 10628 tablet_bootstrap.cc:654] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.043478 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:24.045219 10750 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.045341 10785 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.046080 10785 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.058497 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:24.071086 10809 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.072078 10511 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.071871 10809 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.077816 10627 tablet_bootstrap.cc:654] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.085433 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:24.089838 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:24.091593 10747 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.096192 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:24.096918 10587 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.092742 10809 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:24.100399 10785 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.124795 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:09:24.112924 10553 leader_election.cc:343] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.131336 10587 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:24.128894 10478 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:24.145051 10629 tablet_bootstrap.cc:492] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:24.145757 10629 ts_tablet_manager.cc:1403] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.211s	user 0.017s	sys 0.025s
I20260709 12:09:24.145490 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.113718 10837 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:09:24.133915 10703 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.154202 10478 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:24.155926 10837 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:24.210546 10628 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:24.215443 10628 ts_tablet_manager.cc:1403] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.293s	user 0.022s	sys 0.006s
I20260709 12:09:24.219939 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:24.221051 10587 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:09:24.200402 10703 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.156831 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:24.226846 10627 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:24.227978 10627 ts_tablet_manager.cc:1403] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.277s	user 0.011s	sys 0.014s
W20260709 12:09:24.231657 10479 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:24.232007 10627 raft_consensus.cc:359] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.233022 10627 raft_consensus.cc:385] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.233381 10627 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.163609 10438 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:09:24.166127 10552 leader_election.cc:343] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.167464 10634 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.068s
I20260709 12:09:24.235175 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:24.235226 10627 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.236585 10809 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:24.236873 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70815.2us
I20260709 12:09:24.166783 10750 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.248081 10629 raft_consensus.cc:359] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.249245 10629 raft_consensus.cc:385] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.249625 10629 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
W20260709 12:09:24.253580 10552 consensus_peers.cc:597] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:24.250352 10629 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.261479 10809 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.262771 10629 ts_tablet_manager.cc:1434] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.117s	user 0.016s	sys 0.005s
I20260709 12:09:24.263536 10634 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.264089 10629 tablet_bootstrap.cc:492] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:24.277582 10837 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:24.284648 10629 tablet_bootstrap.cc:654] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.288511 10627 ts_tablet_manager.cc:1434] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 12:09:24.293252 10627 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:24.309959 10628 raft_consensus.cc:359] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.310822 10628 raft_consensus.cc:385] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.311369 10628 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.312471 10628 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.326668 10628 ts_tablet_manager.cc:1434] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.002s
I20260709 12:09:24.279386 10837 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.342558 10785 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.348026 10750 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.348656 10750 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.352903 10839 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.361054 10628 tablet_bootstrap.cc:492] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:24.353225 10750 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.357643 10839 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.367285 10839 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.374557 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:24.375622 10587 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:24.379732 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:09:24.389770 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:24.390225 10703 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.393558 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:24.394361 10585 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:24.396770 10478 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:24.385972 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:24.398149 10437 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:24.399140 10839 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.399545 10839 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.399878 10839 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.381863 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.403462 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:24.406535 10703 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.421815 10703 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.422370 10703 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.426088 10336 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.455439 10628 tablet_bootstrap.cc:654] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.449317 10785 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.463201 10785 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.467303 10785 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.467976 10629 tablet_bootstrap.cc:492] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:24.468466 10629 ts_tablet_manager.cc:1403] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.205s	user 0.013s	sys 0.015s
I20260709 12:09:24.475507 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:24.482352 10836 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:24.482602 10851 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:09:24.483286 10836 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.483894 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:24.485306 10629 raft_consensus.cc:359] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.488566 10629 raft_consensus.cc:385] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.489027 10629 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.488021 10703 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.490371 10629 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.498184 10627 tablet_bootstrap.cc:654] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:24.499140 10553 consensus_peers.cc:597] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:24.498250 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:24.508025 10742 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.508915 10836 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.513392 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.513478 10634 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:09:24.526530 10703 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.535725 10839 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.537791 10629 ts_tablet_manager.cc:1434] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.000s
I20260709 12:09:24.538137 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:24.538288 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:24.539142 10511 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:09:24.539623 10629 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:24.546861 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:24.596549 10584 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.618268 10628 tablet_bootstrap.cc:492] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:24.619621 10628 ts_tablet_manager.cc:1403] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.259s	user 0.012s	sys 0.018s
I20260709 12:09:24.621714 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:24.624019 10839 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.628382 10628 raft_consensus.cc:359] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.630174 10628 raft_consensus.cc:385] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.632061 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:24.634070 10437 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:09:24.636181 10627 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:24.630645 10628 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.639492 10627 ts_tablet_manager.cc:1403] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.350s	user 0.023s	sys 0.004s
I20260709 12:09:24.639493 10628 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.643055 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:24.643736 10629 tablet_bootstrap.cc:654] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.656339 10627 raft_consensus.cc:359] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.661290 10627 raft_consensus.cc:385] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.661578 10627 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.662596 10628 ts_tablet_manager.cc:1434] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.042s	user 0.013s	sys 0.002s
I20260709 12:09:24.667296 10628 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:24.667433 10478 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:24.668687 10839 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.670979 10627 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.674932 10858 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.684441 10512 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.695525 10627 ts_tablet_manager.cc:1434] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.001s
I20260709 12:09:24.695883 10584 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:24.696808 10809 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.697705 10809 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.703497 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:24.705760 10663 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:24.706506 10663 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:24.705790 10628 tablet_bootstrap.cc:654] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.697636 10627 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:24.712352 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.707751 10663 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.716490 10433 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:24.721338 10552 leader_election.cc:343] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.708333 10809 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:24.722610 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:24.731582 10553 leader_election.cc:343] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.732339 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:24.734689 10809 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.743111 10629 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:24.753674 10629 ts_tablet_manager.cc:1403] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.214s	user 0.025s	sys 0.026s
I20260709 12:09:24.775178 10629 raft_consensus.cc:359] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.776150 10629 raft_consensus.cc:385] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.776448 10629 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.777264 10629 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.786825 10628 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:24.787623 10628 ts_tablet_manager.cc:1403] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.121s	user 0.020s	sys 0.005s
I20260709 12:09:24.791517 10629 ts_tablet_manager.cc:1434] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.007s
I20260709 12:09:24.791692 10627 tablet_bootstrap.cc:654] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.792523 10628 raft_consensus.cc:359] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.793123 10629 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:24.793469 10628 raft_consensus.cc:385] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.792541 10663 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.793891 10628 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.794752 10663 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.795445 10628 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.797247 10628 ts_tablet_manager.cc:1434] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:09:24.799849 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:24.800294 10628 tablet_bootstrap.cc:492] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:24.800716 10584 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:24.802346 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:24.803483 10703 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.803870 10703 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.804672 10703 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.805603 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.797219 10663 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:24.809051 10403 leader_election.cc:343] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.817890 10629 tablet_bootstrap.cc:654] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.820832 10628 tablet_bootstrap.cc:654] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.833424 10627 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:24.833936 10627 ts_tablet_manager.cc:1403] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.137s	user 0.017s	sys 0.014s
I20260709 12:09:24.848941 10627 raft_consensus.cc:359] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.851680 10627 raft_consensus.cc:385] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.852030 10627 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.852914 10627 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.857601 10837 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.860318 10703 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.864434 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:24.865983 10627 ts_tablet_manager.cc:1434] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.002s
I20260709 12:09:24.865507 10334 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.866362 10512 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.867775 10627 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:24.887362 10634 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.892284 10627 tablet_bootstrap.cc:654] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.892681 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:24.892299 10634 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.893332 10584 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.895696 10628 tablet_bootstrap.cc:492] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:24.896384 10628 ts_tablet_manager.cc:1403] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.096s	user 0.008s	sys 0.015s
I20260709 12:09:24.896373 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:24.897428 10433 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:24.899179 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:24.900168 10809 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.900053 10628 raft_consensus.cc:359] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.900620 10809 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.900837 10628 raft_consensus.cc:385] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.901024 10809 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.901197 10628 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.902016 10584 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:24.903410 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:24.904420 10663 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:24.904954 10663 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:24.905800 10629 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:24.906852 10629 ts_tablet_manager.cc:1403] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.114s	user 0.020s	sys 0.003s
I20260709 12:09:24.902335 10628 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.906651 10663 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.908536 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.909106 10634 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:24.909893 10628 ts_tablet_manager.cc:1434] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:09:24.920624 10628 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
W20260709 12:09:24.910969 10552 leader_election.cc:343] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.932296 10628 tablet_bootstrap.cc:654] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.911347 10703 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:24.912048 10705 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.916914 10629 raft_consensus.cc:359] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.935969 10629 raft_consensus.cc:385] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.935575 10705 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.936306 10629 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.937209 10629 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.938453 10705 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:24.941438 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:24.941586 10837 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:24.942163 10433 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:24.948552 10809 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.953788 10629 ts_tablet_manager.cc:1434] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:09:24.883196 10512 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:24.963142 10705 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.946120 10334 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.963737 10705 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.965577 10627 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:24.965711 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:24.966511 10627 ts_tablet_manager.cc:1403] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.099s	user 0.020s	sys 0.006s
I20260709 12:09:24.972476 10705 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:24.973281 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:24.974479 10705 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.974903 10705 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.975643 10705 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.977118 10809 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:24.971115 10627 raft_consensus.cc:359] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.981003 10808 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.981334 10627 raft_consensus.cc:385] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.966511 10513 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:24.981827 10627 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.981647 10808 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.983498 10627 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:24.994385 10627 ts_tablet_manager.cc:1434] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:09:24.996343 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:24.997072 10627 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:24.998342 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:25.004884 10552 leader_election.cc:343] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:25.005792 10552 leader_election.cc:343] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.014185 10629 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:25.011483 10808 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:25.046706 10627 tablet_bootstrap.cc:654] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.082283 10628 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:25.083068 10628 ts_tablet_manager.cc:1403] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.163s	user 0.022s	sys 0.018s
I20260709 12:09:25.083978 10629 tablet_bootstrap.cc:654] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.087455 10628 raft_consensus.cc:359] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.088243 10628 raft_consensus.cc:385] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.088507 10628 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.091434 10628 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.093422 10628 ts_tablet_manager.cc:1434] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:09:25.094636 10628 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:25.119993 10628 tablet_bootstrap.cc:654] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.132817 10629 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:25.133472 10629 ts_tablet_manager.cc:1403] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.009s
I20260709 12:09:25.136919 10629 raft_consensus.cc:359] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.137784 10629 raft_consensus.cc:385] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.138156 10629 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
W20260709 12:09:25.144902 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 168366us
I20260709 12:09:25.147497 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:25.148228 10438 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.151247 10629 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.157765 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:25.150684 10705 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.180056 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:25.184646 10553 leader_election.cc:343] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.202214 10837 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.202944 10837 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.207017 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:25.207873 10435 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:25.210095 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:25.211125 10839 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.211558 10839 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.211941 10839 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.217278 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:25.218150 10584 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:25.227130 10837 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:25.227339 10629 ts_tablet_manager.cc:1434] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.000s
I20260709 12:09:25.228740 10629 tablet_bootstrap.cc:492] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:25.231683 10553 leader_election.cc:343] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.232285 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:25.235941 10808 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.259598 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:25.273430 10628 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:25.274123 10628 ts_tablet_manager.cc:1403] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.180s	user 0.041s	sys 0.040s
I20260709 12:09:25.275985 10705 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:25.281117 10628 raft_consensus.cc:359] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.281873 10628 raft_consensus.cc:385] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.282174 10628 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.282928 10628 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.425796 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:25.426900 10435 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.476976 10438 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:25.509088 10629 tablet_bootstrap.cc:654] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.509305 10435 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:25.518326 10884 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:25.522399 10629 tablet_bootstrap.cc:492] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:25.523185 10629 ts_tablet_manager.cc:1403] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.295s	user 0.021s	sys 0.005s
I20260709 12:09:25.526378 10703 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:25.527348 10629 raft_consensus.cc:359] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.528206 10629 raft_consensus.cc:385] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.528546 10629 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.529192 10628 ts_tablet_manager.cc:1434] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.255s	user 0.016s	sys 0.012s
I20260709 12:09:25.529312 10629 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.535813 10629 ts_tablet_manager.cc:1434] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:09:25.537014 10629 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:25.538133 10888 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.540511 10888 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.554971 10628 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:25.559661 10627 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:25.560252 10627 ts_tablet_manager.cc:1403] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.564s	user 0.020s	sys 0.027s
I20260709 12:09:25.551285 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:25.569185 10552 leader_election.cc:343] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.570706 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:25.570569 10627 raft_consensus.cc:359] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.571343 10627 raft_consensus.cc:385] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.571612 10627 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.572459 10627 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.570147 10513 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.590509 10629 tablet_bootstrap.cc:654] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.576828 10627 ts_tablet_manager.cc:1434] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:09:25.594134 10513 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:25.566461 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:25.599454 10627 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:25.595594 10888 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:25.604732 10889 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.613824 10628 tablet_bootstrap.cc:654] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.619982 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:25.622756 10889 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.623406 10889 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:25.234838 10839 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.624408 10889 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.635957 10839 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:25.647459 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:25.648240 10584 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.651578 10893 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.652470 10893 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:25.652842 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:25.668026 10552 leader_election.cc:343] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:25.678627 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 453481us
W20260709 12:09:25.668725 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 107155us
I20260709 12:09:25.654112 10893 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.772295 10889 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.773078 10889 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.775280 10880 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.775956 10880 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.780524 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:25.781422 10586 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:25.784633 10880 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:25.787407 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:25.848469 10889 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:25.851377 10837 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.851992 10837 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.852432 10837 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.925464 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:25.926227 10435 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.936606 10629 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:25.937175 10629 ts_tablet_manager.cc:1403] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.400s	user 0.041s	sys 0.059s
I20260709 12:09:25.949609 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:25.965731 10628 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:25.966331 10628 ts_tablet_manager.cc:1403] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.412s	user 0.018s	sys 0.013s
I20260709 12:09:25.973003 10629 raft_consensus.cc:359] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.973958 10629 raft_consensus.cc:385] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.974320 10629 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.975529 10629 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.977661 10629 ts_tablet_manager.cc:1434] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.005s
I20260709 12:09:25.978969 10629 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:25.980234 10628 raft_consensus.cc:359] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.980938 10628 raft_consensus.cc:385] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.981225 10628 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.981971 10628 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:25.984258 10435 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:25.986582 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:25.990736 10628 ts_tablet_manager.cc:1434] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 12:09:25.993517 10584 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:25.996068 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:25.998737 10839 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.999475 10839 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:26.000629 10839 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.004691 10628 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:26.021514 10627 tablet_bootstrap.cc:654] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:26.030055 10552 leader_election.cc:343] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:26.011413 10553 leader_election.cc:343] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.035085 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.036388 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:09:26.009112 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56754.8us
I20260709 12:09:26.037465 10882 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:26.040761 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:26.043414 10882 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.045899 10889 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:26.045305 10837 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.045231 10883 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:09:26.047355 10882 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.049319 10703 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:26.049726 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.049772 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:26.050655 10584 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.050591 10837 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.059202 10892 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.059834 10892 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.064596 10892 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:26.072139 10584 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:26.072542 10336 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:26.076599 10336 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:26.084028 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.088934 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:26.090095 10880 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:26.090617 10880 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:26.091552 10880 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.094156 10629 tablet_bootstrap.cc:654] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:26.098094 10553 leader_election.cc:343] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.098816 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.114634 10627 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:26.115235 10627 ts_tablet_manager.cc:1403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.516s	user 0.049s	sys 0.064s
W20260709 12:09:26.118034 10553 leader_election.cc:343] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.122936 10436 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.123710 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.106495 10893 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:26.132786 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:26.133826 10585 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
W20260709 12:09:26.137084 10479 leader_election.cc:343] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.142030 10628 tablet_bootstrap.cc:654] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.146437 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.147266 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:26.151891 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:09:26.152216 10552 leader_election.cc:343] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.154747 10552 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:09:26.158520 10403 leader_election.cc:343] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.160511 10703 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:26.164206 10891 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:26.162904 10703 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.165731 10703 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.172775 10627 raft_consensus.cc:359] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.174521 10627 raft_consensus.cc:385] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.174832 10627 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.176553 10627 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.178736 10703 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.193316 10334 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:26.202534 10888 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.203156 10888 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:26.199705 10479 leader_election.cc:343] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.183305 10891 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:09:26.213938 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.215363 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:26.215672 10888 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:09:26.217057 10553 leader_election.cc:343] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.217695 10627 ts_tablet_manager.cc:1434] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.000s
I20260709 12:09:26.221153 10627 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:26.218998 10703 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:26.227069 10403 leader_election.cc:343] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.235656 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.236637 10513 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:26.226202 10808 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:09:26.246150 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.248065 10891 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:26.248485 10891 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.248839 10891 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.243966 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:26.254386 10585 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.261464 10891 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.289256 10891 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:26.290434 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:09:26.292688 10553 leader_election.cc:343] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.299911 10629 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:26.300611 10629 ts_tablet_manager.cc:1403] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.322s	user 0.029s	sys 0.023s
I20260709 12:09:26.300987 10628 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:26.301524 10628 ts_tablet_manager.cc:1403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.310s	user 0.019s	sys 0.012s
I20260709 12:09:26.303099 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:26.303824 10513 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.309779 10629 raft_consensus.cc:359] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.310568 10629 raft_consensus.cc:385] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.310858 10629 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.312003 10513 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:26.313344 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.315567 10627 tablet_bootstrap.cc:654] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.317081 10705 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:26.317832 10705 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:26.318764 10705 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.316159 10629 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.325541 10628 raft_consensus.cc:359] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.327386 10628 raft_consensus.cc:385] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.327829 10628 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.313112 10585 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:26.330559 10629 ts_tablet_manager.cc:1434] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 12:09:26.331549 10628 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.337563 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:26.338691 10858 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:26.339397 10858 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:26.341450 10628 ts_tablet_manager.cc:1434] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.005s
I20260709 12:09:26.341897 10629 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:26.340310 10858 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.354463 10628 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:26.356165 10627 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:26.356559 10627 ts_tablet_manager.cc:1403] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.136s	user 0.016s	sys 0.000s
I20260709 12:09:26.359573 10627 raft_consensus.cc:359] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.360718 10627 raft_consensus.cc:385] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.361982 10627 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.363025 10627 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.365592 10627 ts_tablet_manager.cc:1434] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:09:26.360601 10705 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.378511 10705 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.384332 10627 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:26.388082 10880 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:26.400805 10705 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:26.401932 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.403167 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:26.403792 10553 leader_election.cc:343] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.425271 10629 tablet_bootstrap.cc:654] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:26.427230 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001696a0 after lost signal to thread 9264
I20260709 12:09:26.427475 10336 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:09:26.429023 10534 debug-util.cc:398] Leaking SignalData structure 0x7b0800166d40 after lost signal to thread 10370
W20260709 12:09:26.430351 10534 debug-util.cc:398] Leaking SignalData structure 0x7b0800245560 after lost signal to thread 10462
I20260709 12:09:26.430809 10705 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:09:26.431531 10534 debug-util.cc:398] Leaking SignalData structure 0x7b080024b160 after lost signal to thread 10537
W20260709 12:09:26.432698 10534 debug-util.cc:398] Leaking SignalData structure 0x7b080023e6c0 after lost signal to thread 10611
W20260709 12:09:26.436348 10552 leader_election.cc:343] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.436848 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.438700 10705 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:26.444274 10839 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:26.449484 10628 tablet_bootstrap.cc:654] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.469574 10627 tablet_bootstrap.cc:654] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.469699 10334 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:26.496409 10552 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:26.499260 10438 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.529845 10336 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:26.537109 10628 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:26.538002 10628 ts_tablet_manager.cc:1403] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.193s	user 0.012s	sys 0.012s
I20260709 12:09:26.541702 10628 raft_consensus.cc:359] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.542280 10628 raft_consensus.cc:385] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.548393 10703 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.549043 10628 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.549203 10703 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.550657 10628 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.553146 10628 ts_tablet_manager.cc:1434] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:09:26.553598 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.554577 10628 tablet_bootstrap.cc:492] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:26.554936 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:26.555765 10583 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:26.554713 10703 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.563500 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:26.564411 10629 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:26.564354 10900 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:09:26.564815 10900 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.565143 10900 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:26.566814 10628 tablet_bootstrap.cc:654] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.572197 10629 ts_tablet_manager.cc:1403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.233s	user 0.023s	sys 0.018s
I20260709 12:09:26.573565 10880 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:26.573863 10900 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.576805 10435 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:09:26.577700 10629 raft_consensus.cc:359] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.578397 10629 raft_consensus.cc:385] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.578718 10629 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.578584 10880 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:26.579493 10880 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:26.579876 10880 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:26.580690 10880 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.581998 10880 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:26.601728 10628 tablet_bootstrap.cc:492] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:26.602655 10628 ts_tablet_manager.cc:1403] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.048s	user 0.006s	sys 0.011s
I20260709 12:09:26.604199 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:26.611420 10900 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.612823 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:26.579473 10629 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.617504 10583 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:26.620550 10629 ts_tablet_manager.cc:1434] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 12:09:26.622272 10629 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:26.625602 10628 raft_consensus.cc:359] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.628986 10509 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:09:26.626288 10628 raft_consensus.cc:385] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.637733 10628 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.643674 10628 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.649770 10628 ts_tablet_manager.cc:1434] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.003s
I20260709 12:09:26.644160 10583 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:09:26.658857 10628 tablet_bootstrap.cc:492] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:26.658701 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:26.659650 10627 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:26.660177 10627 ts_tablet_manager.cc:1403] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.283s	user 0.023s	sys 0.012s
I20260709 12:09:26.660095 10900 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:26.660774 10900 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:26.661758 10900 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.663261 10627 raft_consensus.cc:359] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.664031 10627 raft_consensus.cc:385] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.664330 10627 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.665150 10627 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.657244 10629 tablet_bootstrap.cc:654] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.672075 10888 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.674330 10888 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.680099 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.682843 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.683853 10627 ts_tablet_manager.cc:1434] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
W20260709 12:09:26.684827 10552 leader_election.cc:343] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:26.685527 10553 leader_election.cc:343] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.686020 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.686968 10893 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:26.685456 10888 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:26.690565 10627 tablet_bootstrap.cc:492] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:26.698726 10628 tablet_bootstrap.cc:654] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.703670 10629 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:26.704209 10629 ts_tablet_manager.cc:1403] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.005s
I20260709 12:09:26.704650 10438 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.717608 10627 tablet_bootstrap.cc:654] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.717531 10629 raft_consensus.cc:359] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.718328 10629 raft_consensus.cc:385] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.718582 10629 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.719238 10629 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.725168 10629 ts_tablet_manager.cc:1434] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:09:26.726456 10629 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:26.733695 10628 tablet_bootstrap.cc:492] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:26.734354 10628 ts_tablet_manager.cc:1403] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.004s
I20260709 12:09:26.737622 10334 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:26.748103 10888 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:26.748443 10628 raft_consensus.cc:359] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.753518 10628 raft_consensus.cc:385] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.753904 10628 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.754834 10628 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.757831 10628 ts_tablet_manager.cc:1434] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.005s
I20260709 12:09:26.762567 10628 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:26.783885 10900 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.784507 10900 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.785233 10629 tablet_bootstrap.cc:654] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.793103 10627 tablet_bootstrap.cc:492] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:26.793627 10627 ts_tablet_manager.cc:1403] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.103s	user 0.028s	sys 0.005s
I20260709 12:09:26.795603 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.796373 10509 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:26.797890 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:26.790813 10900 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.798327 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:26.798720 10583 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:26.799253 10900 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:26.799695 10900 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.800077 10900 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.805060 10888 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:26.808059 10900 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.816900 10900 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.818058 10628 tablet_bootstrap.cc:654] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.821182 10893 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.821930 10893 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.817813 10627 raft_consensus.cc:359] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.826083 10627 raft_consensus.cc:385] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.826439 10627 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.827442 10627 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.830011 10627 ts_tablet_manager.cc:1434] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:09:26.832993 10627 tablet_bootstrap.cc:492] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:26.834836 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:26.835664 10584 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.849655 10893 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:26.852205 10839 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.859185 10839 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.861136 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:26.861907 10900 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:26.870340 10584 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
W20260709 12:09:26.872592 10553 leader_election.cc:343] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.874665 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:26.882436 10900 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:26.867226 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:26.889155 10900 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:26.889581 10509 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.889649 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:26.890319 10587 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:26.890149 10900 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.873510 10839 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.893628 10629 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:26.894099 10629 ts_tablet_manager.cc:1403] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.168s	user 0.020s	sys 0.005s
I20260709 12:09:26.895017 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.896855 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:26.898927 10479 leader_election.cc:343] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.901261 10629 raft_consensus.cc:359] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.902182 10629 raft_consensus.cc:385] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.902503 10629 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
W20260709 12:09:26.903810 10403 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:26.903369 10629 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.905611 10478 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.906734 10629 ts_tablet_manager.cc:1434] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 12:09:26.908069 10629 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:26.907889 10837 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:26.908463 10837 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.908893 10837 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:26.911562 10552 leader_election.cc:343] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.912211 10552 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:26.914383 10705 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:26.929119 10627 tablet_bootstrap.cc:654] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.961643 10837 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.968298 10509 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:26.968763 10858 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.976117 10858 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.977083 10880 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.977869 10880 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.980276 10858 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:26.970201 10334 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:26.988852 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:26.989768 10583 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:26.991214 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:26.992334 10509 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:09:26.995779 10705 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.995994 10628 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:26.996601 10628 ts_tablet_manager.cc:1403] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.238s	user 0.035s	sys 0.033s
I20260709 12:09:26.996424 10705 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.000133 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:27.008710 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:27.043293 10628 raft_consensus.cc:359] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.052700 10628 raft_consensus.cc:385] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.053086 10628 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.053228 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:27.054785 10628 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:26.998222 10880 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.010181 10900 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:27.071457 10705 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:27.071950 10584 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:09:27.075407 10509 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:09:27.053597 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:27.012933 10837 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.055501 10892 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.085331 10509 pending_rounds.cc:85] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 0
I20260709 12:09:27.088024 10892 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.088467 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:27.074717 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:27.079670 10438 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:27.090643 10553 leader_election.cc:343] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.091676 10892 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:27.092813 10583 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.093309 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:27.094908 10553 leader_election.cc:343] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.088070 10509 pending_rounds.cc:107] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:09:27.096086 10509 raft_consensus.cc:2889] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:09:27.095820 10929 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:09:27.099836 10629 tablet_bootstrap.cc:654] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.109153 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:27.107928 10858 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:27.110834 10839 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.109148 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:27.115370 10438 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:27.122584 10628 ts_tablet_manager.cc:1434] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.126s	user 0.010s	sys 0.014s
I20260709 12:09:27.131260 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:27.134615 10552 leader_election.cc:343] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.135169 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:09:27.136443 10552 leader_election.cc:343] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.136623 10888 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.136925 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:27.117599 10900 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:27.139379 10628 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:27.144407 10583 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:27.139217 10900 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:27.148622 10509 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.133024 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:27.155378 10941 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:27.156816 10941 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:27.164491 10941 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.177181 10942 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.190487 10627 tablet_bootstrap.cc:492] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:27.191154 10627 ts_tablet_manager.cc:1403] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.358s	user 0.016s	sys 0.009s
I20260709 12:09:27.194329 10627 raft_consensus.cc:359] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.195101 10627 raft_consensus.cc:385] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.195418 10627 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.196645 10627 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.203595 10629 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:27.204080 10627 ts_tablet_manager.cc:1434] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:09:27.205291 10629 ts_tablet_manager.cc:1403] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.298s	user 0.012s	sys 0.012s
I20260709 12:09:27.207417 10900 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:27.212816 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:27.205374 10627 tablet_bootstrap.cc:492] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:27.229503 10629 raft_consensus.cc:359] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.230501 10629 raft_consensus.cc:385] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.231189 10629 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.234174 10629 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.236640 10629 ts_tablet_manager.cc:1434] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 12:09:27.237684 10629 tablet_bootstrap.cc:492] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:27.239176 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:27.243242 10900 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.244568 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:27.245242 10584 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:27.244486 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:27.261188 10584 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:09:27.263032 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:27.265144 10900 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:27.269536 10900 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:27.270535 10629 tablet_bootstrap.cc:654] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.270782 10900 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.272493 10627 tablet_bootstrap.cc:654] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.273638 10628 tablet_bootstrap.cc:654] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.293718 10929 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:27.320149 10629 tablet_bootstrap.cc:492] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:27.320730 10629 ts_tablet_manager.cc:1403] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.011s
I20260709 12:09:27.325984 10629 raft_consensus.cc:359] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.327179 10629 raft_consensus.cc:385] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.327493 10629 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.338325 10628 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:27.339015 10628 ts_tablet_manager.cc:1403] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.204s	user 0.016s	sys 0.012s
I20260709 12:09:27.345050 10628 raft_consensus.cc:359] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.346797 10628 raft_consensus.cc:385] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.347252 10628 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.349442 10900 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:27.329165 10629 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.352444 10629 ts_tablet_manager.cc:1434] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.005s
I20260709 12:09:27.352036 10628 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.358656 10628 ts_tablet_manager.cc:1434] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:09:27.360522 10629 tablet_bootstrap.cc:492] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:27.368911 10628 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:27.376720 10942 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.378417 10942 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.407121 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:27.413066 10942 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:27.415694 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:27.416884 10627 tablet_bootstrap.cc:492] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:27.417457 10627 ts_tablet_manager.cc:1403] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.212s	user 0.010s	sys 0.019s
W20260709 12:09:27.417902 10553 leader_election.cc:343] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.419726 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:27.423851 10552 leader_election.cc:343] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.424410 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:27.429378 10634 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.448144 10510 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.481204 10941 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:27.482511 10629 tablet_bootstrap.cc:654] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.482061 10941 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.457410 10627 raft_consensus.cc:359] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.485627 10627 raft_consensus.cc:385] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.485999 10627 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.494460 10628 tablet_bootstrap.cc:654] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.498513 10627 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.513837 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:27.515880 10627 ts_tablet_manager.cc:1434] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.098s	user 0.010s	sys 0.000s
I20260709 12:09:27.524291 10627 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:27.526762 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:27.529063 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:27.530172 10900 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.530557 10839 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:27.530854 10900 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.531451 10809 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.539958 10941 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.532084 10809 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.545109 10809 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:27.547829 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:27.550339 10552 leader_election.cc:343] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.551011 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:27.553282 10553 leader_election.cc:343] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.553705 10553 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:27.556457 10634 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.563668 10900 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.576561 10629 tablet_bootstrap.cc:492] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:27.576475 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:27.577235 10629 ts_tablet_manager.cc:1403] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.217s	user 0.016s	sys 0.007s
I20260709 12:09:27.576525 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:27.578032 10583 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:27.581898 10629 raft_consensus.cc:359] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.583238 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:27.584592 10900 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:27.585001 10900 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:27.585372 10900 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.587175 10627 tablet_bootstrap.cc:654] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.590406 10628 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:27.590958 10628 ts_tablet_manager.cc:1403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.223s	user 0.024s	sys 0.016s
I20260709 12:09:27.601437 10628 raft_consensus.cc:359] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.607043 10628 raft_consensus.cc:385] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.607396 10628 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.608125 10628 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:27.609275 10403 leader_election.cc:343] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.610572 10628 ts_tablet_manager.cc:1434] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.000s
I20260709 12:09:27.612767 10438 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:09:27.614601 10839 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:27.615532 10839 consensus_queue.cc:1243] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 12:09:27.583045 10629 raft_consensus.cc:385] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.617527 10629 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.617857 10839 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:27.619881 10628 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:27.621500 10839 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:27.622035 10629 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.627178 10839 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.638846 10839 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:27.628990 10627 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:27.640651 10627 ts_tablet_manager.cc:1403] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.117s	user 0.016s	sys 0.000s
I20260709 12:09:27.647980 10628 tablet_bootstrap.cc:654] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.656085 10629 ts_tablet_manager.cc:1434] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.078s	user 0.008s	sys 0.000s
I20260709 12:09:27.634022 10900 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.644594 10627 raft_consensus.cc:359] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.681082 10627 raft_consensus.cc:385] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.681457 10627 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.682377 10627 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.684643 10627 ts_tablet_manager.cc:1434] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:09:27.688323 10628 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:27.689999 10628 ts_tablet_manager.cc:1403] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.002s
I20260709 12:09:27.694000 10628 raft_consensus.cc:359] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.699326 10900 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.695844 10627 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:27.712004 10627 tablet_bootstrap.cc:654] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.679055 10629 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:27.694775 10628 raft_consensus.cc:385] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.719328 10628 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.703083 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:27.702924 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:27.720821 10513 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.720638 10628 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.723757 10628 ts_tablet_manager.cc:1434] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.003s
I20260709 12:09:27.724805 10629 tablet_bootstrap.cc:654] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.728220 10628 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:27.733511 10583 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.738068 10513 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:27.747169 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:27.749902 10929 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:27.750705 10929 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:27.752015 10929 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.753508 10627 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:27.754050 10627 ts_tablet_manager.cc:1403] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.005s
I20260709 12:09:27.768402 10629 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:27.769235 10629 ts_tablet_manager.cc:1403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.003s
I20260709 12:09:27.771883 10628 tablet_bootstrap.cc:654] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.777045 10629 raft_consensus.cc:359] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.786401 10629 raft_consensus.cc:385] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.786015 10627 raft_consensus.cc:359] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.786682 10629 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.788388 10513 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:09:27.787540 10334 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:27.790303 10513 pending_rounds.cc:85] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 0
I20260709 12:09:27.790664 10513 pending_rounds.cc:107] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:09:27.790961 10513 raft_consensus.cc:2889] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:09:27.787559 10629 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.794239 10583 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:27.795169 10629 ts_tablet_manager.cc:1434] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 12:09:27.794456 10900 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:27.796371 10629 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:27.786748 10627 raft_consensus.cc:385] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.799798 10627 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.811470 10627 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.814261 10627 ts_tablet_manager.cc:1434] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.004s
I20260709 12:09:27.815327 10627 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:27.815600 10900 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.816331 10900 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.820713 10900 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.826146 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:27.832540 10585 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:27.828639 10587 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:09:27.831537 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:27.833606 10513 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:27.835770 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:27.837554 10703 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:27.837893 10703 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:27.838201 10703 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.836496 10900 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:09:27.852948 10703 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.858130 10703 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.859687 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:27.860584 10585 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.862231 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:27.863253 10510 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.868938 10629 tablet_bootstrap.cc:654] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.870126 10627 tablet_bootstrap.cc:654] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.884891 10585 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:27.886844 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:27.887807 10900 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:27.888224 10628 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:27.888553 10900 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:27.888868 10628 ts_tablet_manager.cc:1403] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.161s	user 0.017s	sys 0.004s
I20260709 12:09:27.890053 10900 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.909345 10627 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:27.910022 10627 ts_tablet_manager.cc:1403] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.095s	user 0.008s	sys 0.012s
I20260709 12:09:27.915855 10628 raft_consensus.cc:359] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.916596 10628 raft_consensus.cc:385] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.916877 10628 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.917632 10628 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.898856 10510 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:27.920567 10628 ts_tablet_manager.cc:1434] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:09:27.944494 10628 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:27.967453 10335 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:27.970976 10858 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.979678 10858 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.981501 10628 tablet_bootstrap.cc:654] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.987847 10858 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:27.989873 10629 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:27.991516 10629 ts_tablet_manager.cc:1403] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.195s	user 0.027s	sys 0.046s
I20260709 12:09:27.992126 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:27.992372 10627 raft_consensus.cc:359] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:27.997552 10627 raft_consensus.cc:385] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.997996 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:27.998766 10587 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:27.998032 10627 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.009925 10627 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.012989 10627 ts_tablet_manager.cc:1434] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.005s
I20260709 12:09:28.023288 10627 tablet_bootstrap.cc:492] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:28.015134 10634 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.020191 10628 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:28.014252 10629 raft_consensus.cc:359] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.024139 10628 ts_tablet_manager.cc:1403] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.013s
I20260709 12:09:28.024317 10629 raft_consensus.cc:385] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.023931 10634 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.024633 10629 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.026034 10629 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.027798 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:28.028115 10628 raft_consensus.cc:359] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.028846 10628 raft_consensus.cc:385] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.029209 10628 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
W20260709 12:09:28.030019 10552 leader_election.cc:343] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.033035 10634 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:28.023176 10900 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.034010 10900 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.057744 10629 ts_tablet_manager.cc:1434] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.064s	user 0.002s	sys 0.005s
I20260709 12:09:28.059099 10629 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:28.059887 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:28.060927 10858 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.061331 10858 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.061708 10858 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.061748 10628 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.063735 10628 ts_tablet_manager.cc:1434] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:09:28.064611 10809 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.066144 10628 tablet_bootstrap.cc:492] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
W20260709 12:09:28.071734 10403 leader_election.cc:343] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.071743 10809 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.081746 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:28.084160 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:28.085367 10628 tablet_bootstrap.cc:654] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.087360 10634 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.087828 10634 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:28.088244 10553 leader_election.cc:343] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.088820 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.089713 10893 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.091213 10629 tablet_bootstrap.cc:654] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.044093 10900 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:28.093420 10403 leader_election.cc:343] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.093768 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:28.094465 10583 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:28.096493 10627 tablet_bootstrap.cc:654] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.099957 10634 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:28.100739 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:28.101727 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:28.106155 10552 leader_election.cc:343] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.106971 10940 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.107465 10940 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.107820 10940 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.105124 10858 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.111833 10858 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.119601 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:28.120419 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:28.120667 10839 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.120498 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:28.121145 10583 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.121196 10839 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:28.122104 10403 leader_election.cc:343] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.129002 10839 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.130970 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:28.131062 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:28.131701 10438 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:28.132139 10587 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:28.139109 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:28.146337 10839 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.146924 10839 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.151564 10941 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.152388 10839 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.153093 10941 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.156621 10941 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.159070 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:28.161775 10479 leader_election.cc:343] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:28.165973 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51014.4us
I20260709 12:09:28.166607 10629 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
W20260709 12:09:28.167363 10553 leader_election.cc:343] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.167421 10629 ts_tablet_manager.cc:1403] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.109s	user 0.011s	sys 0.009s
I20260709 12:09:28.167810 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.163180 10893 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.186749 10628 tablet_bootstrap.cc:492] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:28.187482 10628 ts_tablet_manager.cc:1403] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.122s	user 0.014s	sys 0.010s
I20260709 12:09:28.187745 10627 tablet_bootstrap.cc:492] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:28.188318 10627 ts_tablet_manager.cc:1403] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.165s	user 0.022s	sys 0.015s
I20260709 12:09:28.171898 10973 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.193487 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:28.194175 10587 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:28.178637 10940 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.184773 10893 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.204427 10628 raft_consensus.cc:359] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.205546 10628 raft_consensus.cc:385] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.205833 10628 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.207940 10627 raft_consensus.cc:359] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.208495 10627 raft_consensus.cc:385] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.208810 10627 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.209448 10627 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.209668 10839 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.212709 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:28.213315 10438 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.217815 10900 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:28.218259 10629 raft_consensus.cc:359] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.220577 10893 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:28.224107 10703 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:28.218945 10629 raft_consensus.cc:385] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.227181 10629 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.207569 10628 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.228699 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:28.229099 10940 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.229187 10627 ts_tablet_manager.cc:1434] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.001s
W20260709 12:09:28.230501 10553 leader_election.cc:343] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.230564 10478 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.230712 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:28.231403 10510 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.231606 10880 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.231866 10627 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:28.231999 10880 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.232419 10880 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.234634 10438 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:28.236289 10629 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.238430 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:28.238564 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:28.239286 10941 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.239509 10628 ts_tablet_manager.cc:1434] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.003s
I20260709 12:09:28.240635 10628 tablet_bootstrap.cc:492] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
W20260709 12:09:28.246447 10552 leader_election.cc:343] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.246945 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.251303 10629 ts_tablet_manager.cc:1434] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 12:09:28.253091 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:28.253731 10585 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.255025 10627 tablet_bootstrap.cc:654] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.266667 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:28.268054 10553 leader_election.cc:343] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.251155 10893 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.237452 10583 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:28.278829 10629 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:28.263619 10839 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.297465 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:28.280427 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:28.301072 10586 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.312084 10510 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:28.303589 10929 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.296193 10809 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:28.317392 10628 tablet_bootstrap.cc:654] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.298528 10941 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:28.323469 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:09:28.327735 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:28.304080 10880 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.338754 10940 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.346460 10586 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:28.352365 10929 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:28.467367 10629 tablet_bootstrap.cc:700] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Time spent opening tablet: real 0.126s	user 0.006s	sys 0.001s
I20260709 12:09:28.468295 10629 tablet_bootstrap.cc:654] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.412241 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:28.468165 10929 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.365283 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:28.484591 10985 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.418434 10940 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:28.381202 10880 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.492342 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:28.493009 10587 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:28.472944 10976 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.116s	user 0.005s	sys 0.004s
I20260709 12:09:28.485261 10985 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:28.494711 10479 leader_election.cc:343] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.355468 10585 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
W20260709 12:09:28.510860 10552 leader_election.cc:343] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.512970 10940 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.512411 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.496129 10974 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.430071 10941 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.518754 10974 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.519385 10809 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.527356 10985 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.547787 10587 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:28.573712 10974 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:28.578341 10478 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.586292 10628 tablet_bootstrap.cc:492] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:28.587090 10628 ts_tablet_manager.cc:1403] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.347s	user 0.013s	sys 0.009s
I20260709 12:09:28.600353 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:28.601583 10984 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:09:28.601926 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:09:28.619062 10553 leader_election.cc:343] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.605098 10979 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.604913 10583 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:28.635545 10979 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:28.640771 10979 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.648767 10629 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:28.649328 10629 ts_tablet_manager.cc:1403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.384s	user 0.038s	sys 0.022s
I20260709 12:09:28.652292 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:28.653017 10703 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.653477 10703 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.653900 10703 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.666731 10627 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:28.667516 10627 ts_tablet_manager.cc:1403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.436s	user 0.020s	sys 0.018s
I20260709 12:09:28.675789 10628 raft_consensus.cc:359] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.676587 10628 raft_consensus.cc:385] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.676833 10628 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.680119 10628 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.664252 10984 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 12:09:28.690078 10628 ts_tablet_manager.cc:1434] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.002s
I20260709 12:09:28.707140 10628 tablet_bootstrap.cc:492] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:28.715991 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:28.719532 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 109800us
I20260709 12:09:28.711066 10703 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:28.746105 10552 leader_election.cc:343] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.749553 10336 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:28.757263 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:28.717008 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:28.761103 10973 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.734742 10627 raft_consensus.cc:359] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.762176 10627 raft_consensus.cc:385] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.762569 10627 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.763484 10627 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:28.765208 10403 leader_election.cc:343] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.735069 10629 raft_consensus.cc:359] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.769518 10629 raft_consensus.cc:385] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.769925 10629 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.771289 10629 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.766767 10627 ts_tablet_manager.cc:1434] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.000s
I20260709 12:09:28.774098 10627 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:28.781605 10629 ts_tablet_manager.cc:1434] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.132s	user 0.007s	sys 0.001s
I20260709 12:09:28.785485 10629 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:28.797181 10703 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:28.810734 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:28.816841 10627 tablet_bootstrap.cc:654] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.818435 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:28.819563 10583 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:28.829998 10403 leader_election.cc:343] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.838680 10628 tablet_bootstrap.cc:654] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.843386 10629 tablet_bootstrap.cc:654] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:28.876305 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 253297us
I20260709 12:09:28.881922 10583 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:28.888849 10627 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:28.891819 10627 ts_tablet_manager.cc:1403] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.118s	user 0.026s	sys 0.012s
I20260709 12:09:28.901124 10627 raft_consensus.cc:359] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.902128 10627 raft_consensus.cc:385] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.906615 10627 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.909535 10627 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.919281 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:28.920323 10703 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.920902 10703 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:28.921815 10703 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.930864 10627 ts_tablet_manager.cc:1434] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 12:09:28.932395 10627 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:28.955361 10987 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:28.975132 10628 tablet_bootstrap.cc:492] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:28.975943 10628 ts_tablet_manager.cc:1403] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.270s	user 0.014s	sys 0.009s
I20260709 12:09:28.986750 10978 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:09:28.964718 10983 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:09:28.995026 10628 raft_consensus.cc:359] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:28.996062 10628 raft_consensus.cc:385] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.996366 10628 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.004612 10335 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:29.007084 10335 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:29.020298 10627 tablet_bootstrap.cc:654] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.013242 10987 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:29.012593 10628 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.029630 10628 ts_tablet_manager.cc:1434] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.003s
I20260709 12:09:29.031828 10979 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:29.060277 10628 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:29.068786 10629 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:29.110126 10893 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.111174 10893 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.140801 10629 ts_tablet_manager.cc:1403] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.356s	user 0.032s	sys 0.010s
I20260709 12:09:29.123911 10336 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:29.141467 10880 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.193308 10880 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.214459 10996 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.222114 10977 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.224100 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:29.246201 10629 raft_consensus.cc:359] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.236383 10893 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:29.247064 10629 raft_consensus.cc:385] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.247404 10629 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.248152 10880 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.248189 10629 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.259373 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:29.252848 10977 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.264861 10977 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.269296 10627 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:29.239709 10996 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.274072 10627 ts_tablet_manager.cc:1403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.342s	user 0.007s	sys 0.023s
I20260709 12:09:29.275328 10984 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.275892 10984 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:29.280177 10479 leader_election.cc:343] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.285076 10628 tablet_bootstrap.cc:654] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.283638 10629 ts_tablet_manager.cc:1434] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.132s	user 0.004s	sys 0.005s
I20260709 12:09:29.286808 10629 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:29.279335 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82040.7us
I20260709 12:09:29.320968 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:29.321841 10585 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:29.322063 10984 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.321049 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:29.332563 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.336545 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:29.339289 11018 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:29.339803 11018 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:29.340157 11018 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.323704 10996 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:29.348022 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:29.341204 10972 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.348953 10972 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.353178 10972 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:29.354223 10512 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.355149 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:29.355816 10553 leader_election.cc:343] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.360589 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:29.368000 10586 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:29.369030 10627 raft_consensus.cc:359] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.369791 10627 raft_consensus.cc:385] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.370101 10627 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
W20260709 12:09:29.385430 10553 leader_election.cc:343] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:29.386593 10553 leader_election.cc:343] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.375635 10627 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.397501 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.400902 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:29.343918 10980 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
I20260709 12:09:29.402186 10984 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:29.407361 10984 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:29.407768 10984 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.442858 10627 ts_tablet_manager.cc:1434] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.168s	user 0.003s	sys 0.010s
I20260709 12:09:29.459524 10627 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:29.467726 10628 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:29.468246 10628 ts_tablet_manager.cc:1403] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.408s	user 0.026s	sys 0.013s
W20260709 12:09:29.508952 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:29.525871 10629 tablet_bootstrap.cc:654] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:29.530483 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:09:29.531251 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 101328us
I20260709 12:09:29.544660 10627 tablet_bootstrap.cc:654] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:29.546032 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49613.4us
I20260709 12:09:29.561372 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.586297 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.591321 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.592255 10505 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:29.596442 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.613415 11018 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.615521 10510 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:09:29.619431 10977 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:29.626159 10984 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:29.627056 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 134343us
I20260709 12:09:29.629328 10984 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.605958 10334 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:29.645774 10628 raft_consensus.cc:359] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.646588 10628 raft_consensus.cc:385] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.646869 10628 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.647812 10628 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.650279 10628 ts_tablet_manager.cc:1434] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.182s	user 0.012s	sys 0.000s
W20260709 12:09:29.645211 10552 leader_election.cc:343] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.652508 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.654376 10996 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:29.663167 10552 leader_election.cc:343] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.663695 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.667079 10968 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:29.671496 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 94228.7us
I20260709 12:09:29.678283 11018 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.684553 10929 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.685316 10929 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.688401 10929 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.689534 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:29.690125 10513 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.694757 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:09:29.696846 10479 leader_election.cc:343] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.697422 10628 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:29.698544 10629 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:29.699074 10629 ts_tablet_manager.cc:1403] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.413s	user 0.013s	sys 0.021s
I20260709 12:09:29.709566 10629 raft_consensus.cc:359] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.711022 10629 raft_consensus.cc:385] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.711618 10629 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.706111 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.715085 10629 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.717898 10629 ts_tablet_manager.cc:1434] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:09:29.721462 10975 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.723376 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:29.724400 10629 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:29.724076 10583 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.726701 10627 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:29.731364 10627 ts_tablet_manager.cc:1403] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.279s	user 0.010s	sys 0.010s
I20260709 12:09:29.742638 10627 raft_consensus.cc:359] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.743537 10627 raft_consensus.cc:385] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.743826 10627 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.744570 10627 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.750571 10627 ts_tablet_manager.cc:1434] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:09:29.753525 11000 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.754422 11000 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.755499 10627 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:29.732167 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:29.760126 10513 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:29.759346 11000 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:29.761554 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.742882 10583 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:29.774236 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:29.776432 10403 consensus_peers.cc:597] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:29.776263 10929 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:29.778612 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:29.779362 10703 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:29.779834 10703 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:09:29.780146 10552 leader_election.cc:343] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.780205 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.782757 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:29.780568 10703 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.783429 10586 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:29.784307 10975 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:29.798872 10581 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:29.810668 10628 tablet_bootstrap.cc:654] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.821818 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.832293 10629 tablet_bootstrap.cc:654] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.836397 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:29.837110 10980 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:29.844988 10980 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:09:29.856599 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46463.8us
I20260709 12:09:29.847213 10900 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:29.848688 10980 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.861442 10900 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:29.837244 10929 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:09:29.875218 11030 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.871238 10900 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.890584 10627 tablet_bootstrap.cc:654] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.875870 11030 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.904863 11030 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:29.933684 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 80316.5us
W20260709 12:09:29.936103 10553 leader_election.cc:343] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.936779 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.939412 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:29.935261 10880 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.942155 10880 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.943151 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:29.951177 10880 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:29.952615 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:29.964884 10335 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:29.973327 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:29.971333 11000 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:29.975296 11033 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:09:29.978075 10629 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:29.978591 10629 ts_tablet_manager.cc:1403] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.260s	user 0.012s	sys 0.012s
W20260709 12:09:29.981408 10403 leader_election.cc:343] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.979863 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:29.986528 11022 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:29.989244 10629 raft_consensus.cc:359] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.990202 10629 raft_consensus.cc:385] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.990628 10629 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.991690 10629 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:29.995178 10628 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:29.995873 10628 ts_tablet_manager.cc:1403] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.324s	user 0.021s	sys 0.008s
I20260709 12:09:30.006042 10506 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.017711 10900 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.019233 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:30.020305 10900 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:30.021392 10627 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:30.021886 10627 ts_tablet_manager.cc:1403] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.267s	user 0.017s	sys 0.011s
I20260709 12:09:30.016404 10628 raft_consensus.cc:359] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.023308 10984 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.023557 10628 raft_consensus.cc:385] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.023916 10628 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.024762 10628 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.026832 10628 ts_tablet_manager.cc:1434] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.000s
I20260709 12:09:30.027961 10628 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:30.030771 10629 ts_tablet_manager.cc:1434] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.001s
I20260709 12:09:30.022310 10336 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:30.036623 10627 raft_consensus.cc:359] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.037340 10627 raft_consensus.cc:385] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.037639 10627 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.044973 10431 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.046929 10627 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.049490 10627 ts_tablet_manager.cc:1434] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:09:30.050540 10627 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:30.052126 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:30.052816 10586 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.054701 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:30.055279 10513 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.060286 10629 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:30.065521 10984 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.066118 10984 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.103121 10513 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:30.105822 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:30.104348 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:30.117172 10987 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:30.123241 10987 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:30.125748 10987 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.122579 10984 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:30.144884 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:30.145833 10581 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:09:30.154274 10403 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:30.176328 10629 tablet_bootstrap.cc:654] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.156005 10984 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.179893 10627 tablet_bootstrap.cc:654] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.226104 10628 tablet_bootstrap.cc:654] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.266861 10586 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:30.301738 10809 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.302325 10809 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.304847 10893 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.305270 10809 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.305460 10893 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.307783 10893 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.310791 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:30.313122 10552 leader_election.cc:343] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.317268 10627 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:30.317860 10627 ts_tablet_manager.cc:1403] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.268s	user 0.017s	sys 0.000s
I20260709 12:09:30.309865 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:30.321074 10627 raft_consensus.cc:359] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:30.321446 10552 leader_election.cc:343] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.321780 10627 raft_consensus.cc:385] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.322134 10627 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.323313 10627 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.339598 10627 ts_tablet_manager.cc:1434] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:09:30.343513 10627 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:30.344835 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:30.345961 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:30.340144 10703 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:30.348655 10553 leader_election.cc:343] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.349291 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:09:30.359429 10553 leader_election.cc:343] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.360198 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.333202 10984 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.352841 11000 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.363672 10974 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.363317 10900 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:09:30.388458 10629 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:30.389174 10629 ts_tablet_manager.cc:1403] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.329s	user 0.013s	sys 0.018s
I20260709 12:09:30.409801 10628 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:30.410454 10628 ts_tablet_manager.cc:1403] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.383s	user 0.013s	sys 0.015s
I20260709 12:09:30.412541 10627 tablet_bootstrap.cc:654] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.417711 11017 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:30.420483 10334 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:30.417536 10629 raft_consensus.cc:359] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.422917 10629 raft_consensus.cc:385] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.423246 10629 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.421420 10628 raft_consensus.cc:359] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.424677 10628 raft_consensus.cc:385] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.424970 10628 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.425627 10628 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.429455 10629 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.432263 10629 ts_tablet_manager.cc:1434] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:09:30.433360 10629 tablet_bootstrap.cc:492] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:30.438091 10628 ts_tablet_manager.cc:1434] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260709 12:09:30.439222 10628 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:30.439551 11022 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.440063 11022 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.446622 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:30.448336 10437 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:30.450651 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:30.454118 10809 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.454267 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:30.454708 10809 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.457006 10809 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.459995 10578 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:30.460763 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:30.447841 11022 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:30.465827 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:30.468990 10552 leader_election.cc:343] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.451455 10880 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:30.472262 10880 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:30.472661 10880 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.496447 10629 tablet_bootstrap.cc:654] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.499715 10628 tablet_bootstrap.cc:654] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:30.520543 10553 leader_election.cc:343] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.521149 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.527863 11000 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.546200 10627 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:30.546808 10627 ts_tablet_manager.cc:1403] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.204s	user 0.044s	sys 0.032s
I20260709 12:09:30.552304 11022 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:30.557111 10627 raft_consensus.cc:359] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.557866 10627 raft_consensus.cc:385] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.558202 10627 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.558979 10627 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.579627 10627 ts_tablet_manager.cc:1434] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:09:30.592053 10880 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.594506 10880 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:30.595149 10809 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.595687 10809 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.597975 10809 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.600733 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:30.601410 10437 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.604578 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:30.605221 10578 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.600764 11028 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:09:30.609151 10983 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:30.619275 10627 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:30.619406 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:30.620283 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:30.624287 10552 leader_election.cc:343] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.625064 10628 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:30.625584 10628 ts_tablet_manager.cc:1403] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.187s	user 0.004s	sys 0.012s
W20260709 12:09:30.630071 10553 leader_election.cc:343] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.634171 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.644101 11020 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.680111 10437 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:30.681325 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:30.671341 11020 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.682508 10880 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:30.683064 10880 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:30.682905 11020 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.684026 10880 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.685444 11020 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.680120 10627 tablet_bootstrap.cc:654] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.714236 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:30.716950 10553 leader_election.cc:343] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:30.721414 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:30.719084 11017 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.724833 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:30.705968 10578 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:30.725790 10629 tablet_bootstrap.cc:492] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:30.726519 10629 ts_tablet_manager.cc:1403] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.293s	user 0.013s	sys 0.019s
I20260709 12:09:30.721313 10628 raft_consensus.cc:359] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.725003 11017 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.728828 10628 raft_consensus.cc:385] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.733152 10628 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.731037 11017 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:30.732326 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:30.737334 10479 leader_election.cc:343] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:30.730162 10552 leader_election.cc:343] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.738004 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.738735 11000 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.740722 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:30.741380 10578 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:30.761379 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.764670 11022 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:30.765111 11022 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:30.765470 11022 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.733829 10628 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.785687 10629 raft_consensus.cc:359] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.790592 10629 raft_consensus.cc:385] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.790935 10629 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.791795 10628 ts_tablet_manager.cc:1434] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.166s	user 0.007s	sys 0.007s
I20260709 12:09:30.800019 10628 tablet_bootstrap.cc:492] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:30.791854 10629 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.803098 10974 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.804131 10974 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.807550 10974 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.809481 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:30.811437 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:30.811474 10552 leader_election.cc:343] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:30.816677 10553 leader_election.cc:343] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.817121 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.818221 10974 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.818673 10629 ts_tablet_manager.cc:1434] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.001s
I20260709 12:09:30.821206 11022 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.822427 10629 tablet_bootstrap.cc:492] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:30.827829 10628 tablet_bootstrap.cc:654] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.826864 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:30.836634 10583 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.845543 10335 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:30.826939 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:09:30.852201 10479 leader_election.cc:343] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.857012 10583 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:30.860850 10627 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:30.863112 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.863154 10627 ts_tablet_manager.cc:1403] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.244s	user 0.024s	sys 0.020s
I20260709 12:09:30.864323 11017 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:30.864856 11017 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:30.825488 11022 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:30.866508 11017 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.886534 10629 tablet_bootstrap.cc:654] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.898074 10627 raft_consensus.cc:359] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.898749 10627 raft_consensus.cc:385] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.899092 10627 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.899857 10627 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.905669 10628 tablet_bootstrap.cc:492] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:30.906302 10628 ts_tablet_manager.cc:1403] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.107s	user 0.011s	sys 0.004s
I20260709 12:09:30.910925 10628 raft_consensus.cc:359] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.911702 10628 raft_consensus.cc:385] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.911993 10628 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.912731 10628 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.915891 10335 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:30.919529 10627 ts_tablet_manager.cc:1434] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.055s	user 0.002s	sys 0.006s
I20260709 12:09:30.920737 10627 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:30.923260 10628 ts_tablet_manager.cc:1434] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:09:30.934742 10628 tablet_bootstrap.cc:492] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:30.948045 10628 tablet_bootstrap.cc:654] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.961056 10629 tablet_bootstrap.cc:492] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:30.961683 10629 ts_tablet_manager.cc:1403] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.139s	user 0.007s	sys 0.005s
I20260709 12:09:30.963631 10974 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.964267 10974 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.965410 10629 raft_consensus.cc:359] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:30.966504 10974 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:30.969065 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:30.971611 10627 tablet_bootstrap.cc:654] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.973197 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:30.966667 10629 raft_consensus.cc:385] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.975121 10629 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.975883 10629 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:30.977713 10552 leader_election.cc:343] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.978210 10629 ts_tablet_manager.cc:1434] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
W20260709 12:09:30.978893 10553 leader_election.cc:343] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.979396 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:30.980561 10893 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.982663 10629 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:30.990382 10629 tablet_bootstrap.cc:654] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.999960 10984 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.000615 10984 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.012080 10984 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.012673 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:31.014463 10627 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:31.015050 10627 ts_tablet_manager.cc:1403] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.095s	user 0.017s	sys 0.000s
W20260709 12:09:31.015035 10403 leader_election.cc:343] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.015863 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.016531 10578 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:31.017938 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:31.018679 10627 raft_consensus.cc:359] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.020781 10627 raft_consensus.cc:385] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.021080 10627 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.021457 11030 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:31.021850 11030 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:31.022296 11030 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.025866 10628 tablet_bootstrap.cc:492] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:31.021804 10627 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.043272 10628 ts_tablet_manager.cc:1403] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.109s	user 0.016s	sys 0.009s
I20260709 12:09:31.044049 10627 ts_tablet_manager.cc:1434] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:09:31.045174 10627 tablet_bootstrap.cc:492] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:31.046737 10628 raft_consensus.cc:359] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.047569 10628 raft_consensus.cc:385] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.047907 10628 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.048786 10628 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.064431 11022 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.065053 11022 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.068902 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:31.071480 10479 leader_election.cc:343] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.071719 10628 ts_tablet_manager.cc:1434] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:09:31.072464 11022 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.041759 10629 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:31.074021 10629 ts_tablet_manager.cc:1403] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.009s
I20260709 12:09:31.077105 10629 raft_consensus.cc:359] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.077840 10629 raft_consensus.cc:385] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.078133 10629 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.077361 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.078689 10583 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:31.080818 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.081182 10628 tablet_bootstrap.cc:492] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:31.084293 10900 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.084836 10900 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.081061 10627 tablet_bootstrap.cc:654] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.086900 10900 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.087857 11017 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:31.088346 11017 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:31.088677 11017 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.092911 10893 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.093429 10893 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.078811 10629 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.098241 10893 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:31.100886 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:31.106587 10552 leader_election.cc:343] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.107915 10629 ts_tablet_manager.cc:1434] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:09:31.109053 10629 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:31.116305 11022 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.118374 10628 tablet_bootstrap.cc:654] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.122553 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:31.131673 11022 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.042380 11030 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.145887 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.146785 10583 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
W20260709 12:09:31.148974 10403 leader_election.cc:343] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.151345 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:31.152021 10900 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:31.152405 10900 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:31.152760 10900 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.158941 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:31.161583 10553 leader_election.cc:343] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.162333 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.181782 11030 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.165228 10893 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.184211 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:31.213301 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51788us
W20260709 12:09:31.218912 10403 leader_election.cc:343] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.220357 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:31.221246 10583 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.242746 11017 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.245143 11017 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.253425 10628 tablet_bootstrap.cc:492] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:31.254061 10628 ts_tablet_manager.cc:1403] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.002s
I20260709 12:09:31.254773 10583 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:31.255252 10629 tablet_bootstrap.cc:654] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.263298 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:31.263834 11072 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.264433 11072 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.267109 11072 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:31.267179 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:31.267940 10585 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:31.269145 10479 leader_election.cc:343] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.273519 10900 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.277801 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:31.280558 10552 leader_election.cc:343] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.293835 10585 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:31.284566 10900 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.299108 10627 tablet_bootstrap.cc:492] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:31.299732 10627 ts_tablet_manager.cc:1403] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.255s	user 0.045s	sys 0.029s
I20260709 12:09:31.301249 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.304252 10628 raft_consensus.cc:359] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.307193 10628 raft_consensus.cc:385] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.307590 10628 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.315415 10627 raft_consensus.cc:359] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.316226 10627 raft_consensus.cc:385] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.316507 10627 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.308916 10628 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.304061 11073 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:31.320940 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:31.335860 10583 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.333611 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:31.333863 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:31.317224 10627 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.341337 10628 ts_tablet_manager.cc:1434] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.004s
I20260709 12:09:31.352329 11073 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:31.366087 10628 tablet_bootstrap.cc:492] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Bootstrap starting.
I20260709 12:09:31.366621 11073 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.367834 10629 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:31.368376 10629 ts_tablet_manager.cc:1403] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.260s	user 0.014s	sys 0.026s
I20260709 12:09:31.368747 10627 ts_tablet_manager.cc:1434] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.000s
I20260709 12:09:31.369828 10627 tablet_bootstrap.cc:492] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:31.342651 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:31.371701 11028 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:31.373044 11028 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:31.374063 11028 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.381940 10629 raft_consensus.cc:359] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.382752 10629 raft_consensus.cc:385] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.383081 10629 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.364216 10583 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:31.389305 10629 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.392964 10629 ts_tablet_manager.cc:1434] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:09:31.393666 10627 tablet_bootstrap.cc:654] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.394109 10629 tablet_bootstrap.cc:492] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:31.359539 11071 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.395216 11071 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.399086 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:31.399865 10430 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:31.401250 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:31.402208 11017 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:31.402611 11017 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:31.403035 11017 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:31.403961 10553 leader_election.cc:343] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.406204 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.407222 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.407442 10583 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:31.409564 10628 tablet_bootstrap.cc:654] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.408974 11072 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.403795 11071 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:31.436978 10403 leader_election.cc:343] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.445537 10334 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:31.448330 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:31.449187 10984 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:31.453532 10984 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:31.455832 10984 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.464185 11017 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.471927 10629 tablet_bootstrap.cc:654] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.478400 11073 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.479076 11073 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.481534 11073 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.483248 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.484091 10587 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:09:31.484318 10627 tablet_bootstrap.cc:492] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:31.484905 10627 ts_tablet_manager.cc:1403] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.115s	user 0.013s	sys 0.008s
I20260709 12:09:31.495911 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:31.500674 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.502025 11071 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.486979 11073 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:09:31.508141 10627 raft_consensus.cc:359] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.508962 10627 raft_consensus.cc:385] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.509284 10627 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.509318 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:31.510062 10578 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.510124 10627 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.511188 11017 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.523938 11072 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.524677 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:31.524658 11072 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.525511 10437 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.527040 11072 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:31.528481 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:31.529615 10479 consensus_peers.cc:597] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:09:31.531785 10552 leader_election.cc:343] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.544675 10627 ts_tablet_manager.cc:1434] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.001s
I20260709 12:09:31.553854 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:31.565205 10437 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:31.567171 10627 tablet_bootstrap.cc:492] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:31.567444 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:31.552438 10578 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:31.569713 11017 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:31.570254 11017 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:09:31.573988 10553 leader_election.cc:343] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.575107 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.571188 11017 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.584751 11071 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.585721 11072 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.589881 10629 tablet_bootstrap.cc:492] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:31.590498 10629 ts_tablet_manager.cc:1403] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.197s	user 0.016s	sys 0.003s
I20260709 12:09:31.601589 11028 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.602408 11028 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.611734 11028 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.620632 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:31.622706 10403 leader_election.cc:343] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.623198 10628 tablet_bootstrap.cc:492] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: No bootstrap required, opened a new log
I20260709 12:09:31.596048 10629 raft_consensus.cc:359] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.620133 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.623783 10628 ts_tablet_manager.cc:1403] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Time spent bootstrapping tablet: real 0.258s	user 0.017s	sys 0.008s
I20260709 12:09:31.624301 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:31.623750 10629 raft_consensus.cc:385] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.625519 10629 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.625902 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:31.626842 10983 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:31.627238 10983 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:31.627585 10983 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.630548 10628 raft_consensus.cc:359] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.632367 10628 raft_consensus.cc:385] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.632723 10628 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.633762 10628 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.637360 10628 ts_tablet_manager.cc:1434] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:09:31.630477 10629 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.644629 10629 ts_tablet_manager.cc:1434] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 12:09:31.645781 10629 tablet_bootstrap.cc:492] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:31.646243 10983 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.656242 10335 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:31.675516 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:31.676914 10403 leader_election.cc:343] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.687373 10627 tablet_bootstrap.cc:654] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.687852 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:31.688527 10587 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.717465 10334 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:31.689011 10983 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.724097 10627 tablet_bootstrap.cc:492] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:31.724707 10627 ts_tablet_manager.cc:1403] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.158s	user 0.024s	sys 0.009s
I20260709 12:09:31.730705 10629 tablet_bootstrap.cc:654] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.734736 10627 raft_consensus.cc:359] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.735586 10627 raft_consensus.cc:385] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.735931 10627 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.740566 10335 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:31.745342 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:31.748015 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:31.751268 10627 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.754168 10627 ts_tablet_manager.cc:1434] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 12:09:31.755420 10627 tablet_bootstrap.cc:492] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:31.755542 11018 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.756048 11018 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.757941 11018 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.759335 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:31.759393 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.759622 10984 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:31.760149 10437 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:31.760435 10984 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:31.761322 10984 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.760174 10583 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:31.764150 11017 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.764645 11017 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.767776 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:31.769513 11073 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:31.769894 11073 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:31.770237 11073 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.771111 10627 tablet_bootstrap.cc:654] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.772940 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:31.776389 11000 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.776984 11000 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.778406 11022 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.766779 11017 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.780468 11000 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:31.781162 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:31.782676 10552 leader_election.cc:343] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.784360 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.785135 10587 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:09:31.786154 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.787351 11018 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.791191 11073 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.797750 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:09:31.800776 10553 leader_election.cc:343] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.801250 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:31.803632 10629 tablet_bootstrap.cc:492] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:31.804634 10629 ts_tablet_manager.cc:1403] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.014s
I20260709 12:09:31.803273 10974 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.811432 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:31.813269 10437 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.834474 10437 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
W20260709 12:09:31.836530 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:31.842510 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:31.846714 11022 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:31.847352 11022 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:31.848289 11022 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.851745 10629 raft_consensus.cc:359] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.852543 10629 raft_consensus.cc:385] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.852838 10629 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.857645 10983 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.856909 10629 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.859225 11073 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.864519 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:31.865218 10578 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.865973 10629 ts_tablet_manager.cc:1434] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.001s
I20260709 12:09:31.867744 10627 tablet_bootstrap.cc:492] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:31.865617 10983 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:09:31.868443 10627 ts_tablet_manager.cc:1403] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.113s	user 0.018s	sys 0.000s
I20260709 12:09:31.868727 10629 tablet_bootstrap.cc:492] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
W20260709 12:09:31.883873 10479 consensus_peers.cc:597] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:31.873230 10333 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:31.898840 10627 raft_consensus.cc:359] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.902652 10627 raft_consensus.cc:385] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:31.902946 10627 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:31.905764 11018 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.908499 10578 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:31.909762 10627 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.914676 10627 ts_tablet_manager.cc:1434] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:09:31.917623 10627 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:31.924113 10629 tablet_bootstrap.cc:654] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:31.961380 10984 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.962210 10984 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:31.965698 10984 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:31.975736 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:09:31.980610 10403 leader_election.cc:343] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:31.980474 10900 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.987846 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:31.988917 10578 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:31.986851 10335 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:31.997251 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:32.007344 10984 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:32.008004 10984 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:32.008546 10984 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.035146 10629 tablet_bootstrap.cc:492] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:32.036984 10629 ts_tablet_manager.cc:1403] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.169s	user 0.033s	sys 0.033s
I20260709 12:09:32.042395 10629 raft_consensus.cc:359] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.043522 10629 raft_consensus.cc:385] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.043962 10629 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.044904 10629 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.046388 11038 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:32.047742 11094 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:09:32.047247 11038 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.057899 11038 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:32.069562 11028 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:32.072649 10629 ts_tablet_manager.cc:1434] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 12:09:32.076084 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:32.077091 10510 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:32.079126 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:32.079871 10587 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:32.089172 10984 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.095003 10629 tablet_bootstrap.cc:492] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:32.101434 10984 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:32.107785 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:32.111266 10984 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:32.112929 10984 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:32.113422 10984 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.117571 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:32.118474 10627 tablet_bootstrap.cc:654] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.122151 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:32.122933 10587 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:32.138428 10403 leader_election.cc:343] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:32.184160 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44935.5us
W20260709 12:09:32.183924 11097 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:09:32.195961 10587 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:32.211191 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:32.219630 11030 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:32.220409 11030 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:32.226760 11022 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:32.229678 10629 tablet_bootstrap.cc:654] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.221804 11030 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:32.233388 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 67719.2us
I20260709 12:09:32.261559 11099 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:32.291931 11018 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:32.328922 10984 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.331401 10984 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:32.331938 10403 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:32.340251 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:32.341001 10507 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.350495 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:32.351326 10578 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:32.365713 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 142056us
I20260709 12:09:32.380049 10629 tablet_bootstrap.cc:492] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:32.380740 10629 ts_tablet_manager.cc:1403] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.291s	user 0.011s	sys 0.012s
I20260709 12:09:32.380076 10627 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:32.384542 10627 ts_tablet_manager.cc:1403] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.467s	user 0.031s	sys 0.038s
W20260709 12:09:32.407313 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 141307us
I20260709 12:09:32.424232 10629 raft_consensus.cc:359] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.425073 10629 raft_consensus.cc:385] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.425443 10629 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.434849 11022 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:32.426285 10629 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.444134 10578 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:32.447943 11103 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:32.457430 10629 ts_tablet_manager.cc:1434] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.076s	user 0.010s	sys 0.000s
I20260709 12:09:32.438977 10507 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:32.486104 10627 raft_consensus.cc:359] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.488453 10627 raft_consensus.cc:385] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.488845 10627 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.495177 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:32.495832 10629 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:32.496960 10984 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:32.497783 10984 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:32.499164 10984 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.493717 10627 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.508137 10627 ts_tablet_manager.cc:1434] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.123s	user 0.005s	sys 0.004s
I20260709 12:09:32.509666 10627 tablet_bootstrap.cc:492] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:32.601171 10629 tablet_bootstrap.cc:654] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.629592 10627 tablet_bootstrap.cc:654] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.685844 10627 tablet_bootstrap.cc:492] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:32.686410 10627 ts_tablet_manager.cc:1403] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.177s	user 0.009s	sys 0.015s
I20260709 12:09:32.697481 10629 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:32.698225 10629 ts_tablet_manager.cc:1403] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.203s	user 0.011s	sys 0.014s
I20260709 12:09:32.699504 10627 raft_consensus.cc:359] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.700448 10627 raft_consensus.cc:385] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.700722 10627 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.701422 10627 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.704335 10627 ts_tablet_manager.cc:1434] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 12:09:32.725198 10629 raft_consensus.cc:359] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.725832 10627 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:32.726168 10629 raft_consensus.cc:385] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.726473 10629 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.727296 10629 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.729911 10629 ts_tablet_manager.cc:1434] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.003s
I20260709 12:09:32.731190 10629 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:32.734392 10900 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:32.735101 10900 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.737460 10900 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:32.737357 10334 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:32.753747 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:32.754352 10629 tablet_bootstrap.cc:654] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.754776 10580 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:32.756507 10627 tablet_bootstrap.cc:654] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.758009 10506 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.762144 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:32.763226 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:32.761828 10334 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:32.766089 10900 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:32.767518 10900 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:32.768139 10900 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.774216 10507 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:32.786159 10403 leader_election.cc:343] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:32.798230 10984 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:32.799044 10900 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.806418 10900 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:32.806936 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:09:32.808574 10403 leader_election.cc:343] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:32.811453 10627 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:32.811982 10627 ts_tablet_manager.cc:1403] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.008s
I20260709 12:09:32.812500 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:32.817101 10586 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.821748 10627 raft_consensus.cc:359] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.822824 10627 raft_consensus.cc:385] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.823237 10627 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.824422 10627 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.828835 10627 ts_tablet_manager.cc:1434] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.002s
I20260709 12:09:32.830535 10627 tablet_bootstrap.cc:492] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
I20260709 12:09:32.840554 10629 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:32.841262 10629 ts_tablet_manager.cc:1403] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.006s
I20260709 12:09:32.845719 10629 raft_consensus.cc:359] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.846589 10629 raft_consensus.cc:385] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:32.846949 10629 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:32.847676 10629 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.849618 10629 ts_tablet_manager.cc:1434] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:09:32.860612 10430 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.855759 10436 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:32.861953 10629 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:32.906301 11111 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:32.915071 10627 tablet_bootstrap.cc:654] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.923501 10900 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:32.929936 10384 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:32.930465 10629 tablet_bootstrap.cc:654] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:32.915164 11111 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:32.967525 10586 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:32.969345 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:32.953635 11111 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:32.947975 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:32.989601 10507 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 0.
I20260709 12:09:33.009091 10627 tablet_bootstrap.cc:492] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:33.027527 10627 ts_tablet_manager.cc:1403] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.197s	user 0.020s	sys 0.010s
I20260709 12:09:33.013415 10629 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:33.016094 10552 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:33.031445 10629 ts_tablet_manager.cc:1403] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.170s	user 0.018s	sys 0.007s
I20260709 12:09:33.031635 11111 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:33.032104 11111 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:33.032452 11111 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.035964 10629 raft_consensus.cc:359] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.036828 10629 raft_consensus.cc:385] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:33.037094 10629 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:33.032545 10627 raft_consensus.cc:359] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.038194 10627 raft_consensus.cc:385] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:33.038488 10627 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:33.039314 10629 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.040482 10627 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.044471 10629 ts_tablet_manager.cc:1434] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:09:33.041045 11111 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.045686 10629 tablet_bootstrap.cc:492] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Bootstrap starting.
I20260709 12:09:33.048023 11111 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:33.048731 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:33.049444 10507 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.048604 10627 ts_tablet_manager.cc:1434] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.002s
I20260709 12:09:33.054174 10627 tablet_bootstrap.cc:492] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Bootstrap starting.
W20260709 12:09:33.058707 11132 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.082s	user 0.005s	sys 0.000s
I20260709 12:09:33.071177 10629 tablet_bootstrap.cc:654] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:33.071381 10507 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:33.062875 10627 tablet_bootstrap.cc:654] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:33.072878 10552 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:33.074760 11106 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:09:33.075371 10553 leader_election.cc:343] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:33.075984 11106 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:33.077306 11106 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.105840 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:33.124382 10900 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:33.125010 10900 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:33.126034 10900 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.169492 10984 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:33.180308 10627 tablet_bootstrap.cc:492] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: No bootstrap required, opened a new log
I20260709 12:09:33.181224 10627 ts_tablet_manager.cc:1403] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent bootstrapping tablet: real 0.127s	user 0.017s	sys 0.008s
I20260709 12:09:33.243206 10629 tablet_bootstrap.cc:492] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: No bootstrap required, opened a new log
I20260709 12:09:33.243922 10629 ts_tablet_manager.cc:1403] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Time spent bootstrapping tablet: real 0.198s	user 0.021s	sys 0.014s
W20260709 12:09:33.261921 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66642.1us
W20260709 12:09:33.278400 11138 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:09:33.314226 10629 raft_consensus.cc:359] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.315174 10629 raft_consensus.cc:385] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:33.315563 10627 raft_consensus.cc:359] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.316679 10627 raft_consensus.cc:385] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:33.316977 10627 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Initialized, Role: FOLLOWER
I20260709 12:09:33.315500 10629 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Initialized, Role: FOLLOWER
I20260709 12:09:33.318054 11137 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:33.318713 11137 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.318868 10627 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.320256 10629 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.323750 10629 ts_tablet_manager.cc:1434] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.004s
I20260709 12:09:33.325398 10627 ts_tablet_manager.cc:1434] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent starting tablet: real 0.144s	user 0.008s	sys 0.006s
I20260709 12:09:33.338424 11137 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:33.344437 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70802.6us
I20260709 12:09:33.347323 10334 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:33.356767 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:33.357527 10438 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.375440 10438 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:09:33.351092 11137 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
W20260709 12:09:33.402865 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 111503us
I20260709 12:09:33.419481 11149 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:33.420262 11149 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.426349 11028 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:33.427029 11028 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.429239 11028 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:33.439324 11149 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:33.439819 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:33.441363 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:33.442042 10510 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:09:33.442405 10586 raft_consensus.cc:2393] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 1: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:09:33.440359 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:33.446710 11149 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:33.447340 11149 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.452472 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:33.454967 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:33.455746 10432 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:33.456473 10586 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 0.
I20260709 12:09:33.456554 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:09:33.460449 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:33.460999 11149 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:33.461675 11141 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:33.462080 11141 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:33.462441 11141 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.470113 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:09:33.480665 10333 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:33.482082 11022 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.494088 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:09:33.495340 10580 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 0.
I20260709 12:09:33.491129 11139 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:09:33.503808 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:33.556238 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:33.557499 11143 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:33.589210 11022 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:33.603171 11141 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.604782 10510 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 1: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:09:33.617560 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 58542.8us
I20260709 12:09:33.620260 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:33.621068 10432 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.629179 11141 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:33.643254 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:33.639303 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:33.650301 10580 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:33.658440 10432 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:33.659310 10900 raft_consensus.cc:2764] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:09:33.663035 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:33.666316 11073 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:33.674141 11073 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:33.675343 11073 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:33.685627 11028 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:33.711520 10580 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:09:33.735422 11106 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:33.746896 10334 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:33.759464 11038 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:34.109117 11141 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:34.159090 11103 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:34.292799  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:34.304486 11166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:34.324455 11167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:34.357537  9261 server_base.cc:1061] running on GCE node
W20260709 12:09:34.360122 11170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:34.362224  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:34.362711  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:34.363122  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783598974363096 us; error 0 us; skew 500 ppm
I20260709 12:09:34.364264  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:34.376708  9261 webserver.cc:533] Webserver started at http://127.9.11.68:36939/ using document root <none> and password file <none>
I20260709 12:09:34.377461  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:34.377772  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:34.378218  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:34.379925  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-3-root/instance:
uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
format_stamp: "Formatted at 2026-07-09 12:09:34 on dist-test-slave-k289"
I20260709 12:09:34.386483  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:09:34.391638 11174 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:34.393018  9261 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:09:34.393402  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-3-root
uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
format_stamp: "Formatted at 2026-07-09 12:09:34 on dist-test-slave-k289"
I20260709 12:09:34.393895  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:34.434237  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:34.437436  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:34.455642  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:34.467123  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:34.467706  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:09:34.468187  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:34.468556  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
W20260709 12:09:34.792606 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48144.9us
W20260709 12:09:35.718587 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 125573us
I20260709 12:09:35.775959  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.68:40987
I20260709 12:09:35.776101 11248 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.68:40987 every 8 connection(s)
I20260709 12:09:35.803279  9261 auto_leader_rebalancer-test.cc:439] add a tserver: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:35.835546 11249 heartbeater.cc:344] Connected to a master server at 127.9.11.126:42677
I20260709 12:09:35.836180 11249 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:35.837393 11249 heartbeater.cc:507] Master 127.9.11.126:42677 requested a full tablet report, sending...
I20260709 12:09:35.840322 10333 ts_manager.cc:194] Registered new tserver with Master: 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:35.842666 10333 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45398
I20260709 12:09:36.554723 10438 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:36.596051 10580 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:36.599696 10506 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:36.620455 11190 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:36.622587 10404 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5d55f0ab8c4445abbbca950ea76dca1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:36.625644 11190 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:36.645161 11194 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.646747 10586 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.673851 10510 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.690891 10333 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.776958 10580 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:36.847688 11249 heartbeater.cc:499] Master 127.9.11.126:42677 was elected leader, sending a full tablet report...
I20260709 12:09:36.893676 10506 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:36.896498 11144 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:36.916070 10437 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:36.921092 11144 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:36.930335 10553 consensus_peers.cc:597] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0bb77704fc9403481da02083315dbb1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:36.936843 11197 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.941432 10437 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.950096 10506 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.965435 10333 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:36.991873 10437 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:37.014555 10404 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4642a95b220c4ecea8115de1fa98954e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:37.016067 10585 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:37.015947 10506 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:37.021445 11194 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:37.024307 11194 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:37.056559 11194 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.068814 10580 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.084452 10506 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.093001 10333 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:37.100741 11276 ts_tablet_manager.cc:933] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:37.193454 11276 tablet_copy_client.cc:323] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:09:37.207687 10508 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:37.228402 10580 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:37.233917 11149 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:37.253037 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet f5d55f0ab8c4445abbbca950ea76dca1 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:09:37.253994 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet f5d55f0ab8c4445abbbca950ea76dca1 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-f5d55f0ab8c4445abbbca950ea76dca1
I20260709 12:09:37.269613 10448 tablet_copy_source_session.cc:215] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:37.272302 10431 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:37.276392 10585 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.285192 11276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d55f0ab8c4445abbbca950ea76dca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:37.273800 11103 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.328688 11276 tablet_copy_client.cc:806] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:37.323787 11284 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:09:37.331908 11276 tablet_copy_client.cc:670] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:37.330808 10333 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.344237 11276 tablet_copy_client.cc:538] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:37.387812 11285 ts_tablet_manager.cc:933] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:37.459337 11276 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:09:37.467511 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 145081us
I20260709 12:09:37.472604 11285 tablet_copy_client.cc:323] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
W20260709 12:09:37.513326 10479 consensus_peers.cc:597] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80de1e7bce9c4d34b0583315cafbc371. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:37.555375 10438 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:37.569380 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet a0bb77704fc9403481da02083315dbb1 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:09:37.570161 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet a0bb77704fc9403481da02083315dbb1 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-a0bb77704fc9403481da02083315dbb1
I20260709 12:09:37.596428 10597 tablet_copy_source_session.cc:215] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:37.605837 11285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0bb77704fc9403481da02083315dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:37.649235 11292 ts_tablet_manager.cc:933] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:37.650813 11276 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:37.675717 11276 tablet_bootstrap.cc:492] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:37.677079 11276 ts_tablet_manager.cc:1403] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.218s	user 0.025s	sys 0.023s
I20260709 12:09:37.676093 11285 tablet_copy_client.cc:806] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:37.682199 11285 tablet_copy_client.cc:670] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:37.694039 11276 raft_consensus.cc:359] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:37.695134 11285 tablet_copy_client.cc:538] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:37.707113 11276 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:37.708122 11276 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:37.711668 11292 tablet_copy_client.cc:323] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:09:37.717772 11276 ts_tablet_manager.cc:1434] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.004s
I20260709 12:09:37.728842 11285 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:37.730327 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet 4642a95b220c4ecea8115de1fa98954e from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:09:37.731086 10447 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-f5d55f0ab8c4445abbbca950ea76dca1 received from {username='slave'} at 127.0.0.1:54450
I20260709 12:09:37.731737 10447 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-f5d55f0ab8c4445abbbca950ea76dca1 on tablet f5d55f0ab8c4445abbbca950ea76dca1 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:37.732683 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet 4642a95b220c4ecea8115de1fa98954e from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-4642a95b220c4ecea8115de1fa98954e
I20260709 12:09:37.744325 10448 tablet_copy_source_session.cc:215] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:37.748109 11292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4642a95b220c4ecea8115de1fa98954e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:37.763450 11276 ts_tablet_manager.cc:933] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:37.787887 11276 tablet_copy_client.cc:323] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:37.849377 11285 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:37.863394 11285 tablet_bootstrap.cc:492] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:37.866291 11285 ts_tablet_manager.cc:1403] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.151s	user 0.028s	sys 0.023s
I20260709 12:09:37.876222 11285 raft_consensus.cc:359] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:37.864461 11292 tablet_copy_client.cc:806] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:37.877698 11292 tablet_copy_client.cc:670] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:37.879756 11285 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:37.880622 11285 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:37.882205 11292 tablet_copy_client.cc:538] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:37.911880 11292 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:37.917517 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 80de1e7bce9c4d34b0583315cafbc371 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:37.918337 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 80de1e7bce9c4d34b0583315cafbc371 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-80de1e7bce9c4d34b0583315cafbc371
I20260709 12:09:37.919684 11285 ts_tablet_manager.cc:1434] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.046s	user 0.001s	sys 0.006s
I20260709 12:09:37.921633 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-a0bb77704fc9403481da02083315dbb1 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:09:37.922129 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-a0bb77704fc9403481da02083315dbb1 on tablet a0bb77704fc9403481da02083315dbb1 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:37.950090 10523 tablet_copy_source_session.cc:215] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:37.955657 11276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80de1e7bce9c4d34b0583315cafbc371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:37.966758 11292 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:37.973939 11276 tablet_copy_client.cc:806] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:37.974803 11276 tablet_copy_client.cc:670] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:37.981454 11276 tablet_copy_client.cc:538] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:37.983702 11292 tablet_bootstrap.cc:492] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:37.985049 11292 ts_tablet_manager.cc:1403] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.073s	user 0.023s	sys 0.005s
I20260709 12:09:37.994798 11292 raft_consensus.cc:359] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:38.003232 11292 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:38.004227 11292 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:38.016736 11292 ts_tablet_manager.cc:1434] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.031s	user 0.001s	sys 0.006s
I20260709 12:09:38.025419 10448 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4642a95b220c4ecea8115de1fa98954e received from {username='slave'} at 127.0.0.1:54450
I20260709 12:09:38.025647 11276 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:38.025980 10448 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4642a95b220c4ecea8115de1fa98954e on tablet 4642a95b220c4ecea8115de1fa98954e with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:38.162921 11276 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:38.164299 11276 tablet_bootstrap.cc:492] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:38.165175 11276 ts_tablet_manager.cc:1403] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.140s	user 0.040s	sys 0.016s
I20260709 12:09:38.171308 11276 raft_consensus.cc:359] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:38.175267 11221 raft_consensus.cc:1217] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:38.195286 11276 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:38.199055 11276 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:38.201965 11276 ts_tablet_manager.cc:1434] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:09:38.204393 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-80de1e7bce9c4d34b0583315cafbc371 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:38.204891 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-80de1e7bce9c4d34b0583315cafbc371 on tablet 80de1e7bce9c4d34b0583315cafbc371 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:38.217619 11221 raft_consensus.cc:1217] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:38.243808 11221 raft_consensus.cc:1217] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:38.276425 11221 raft_consensus.cc:1217] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:09:38.504336 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50246.6us
I20260709 12:09:38.673329 11295 raft_consensus.cc:1064] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:38.676992 11295 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:38.706084 11286 raft_consensus.cc:1064] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:38.709493 11286 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:38.717185 11221 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:38.719266 11256 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:38.804239 10581 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:38.806811 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52255.4us
W20260709 12:09:38.861047 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56643.5us
I20260709 12:09:38.867436 11294 raft_consensus.cc:1064] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:38.870116 11294 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:09:38.877084 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 70565.4us
I20260709 12:09:38.878252 10506 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:38.878213 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83903.2us
I20260709 12:09:38.913326 10581 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:38.932621 11306 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:09:38.968297 11307 raft_consensus.cc:1064] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:38.968750 11221 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:38.972131 11307 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:38.974328 10430 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:38.982144 11221 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:38.983970 11316 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:38.997109 11287 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:09:38.996031 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 65338.5us
I20260709 12:09:39.009406 11315 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:39.005821 11316 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
W20260709 12:09:39.013324 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:09:39.023422 11219 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:39.045619 11223 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:09:39.051118 11294 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:39.057672 10508 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:39.115895 10508 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:39.121203 10581 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:39.093779 11223 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 12:09:39.153774 11321 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:09:39.178954 10437 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:39.173146 11311 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.195822 11272 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:39.210072 11318 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 12:09:39.217749 11323 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:09:39.209523 10430 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.234349 11320 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:09:39.234858 10580 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.257215 10333 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.263193 11314 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.263103 11223 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.282377 11220 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.306352 10508 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.317891 10586 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.453219 11219 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:39.495121 11322 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:09:39.492926 11221 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:39.524029 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 184874us
I20260709 12:09:39.538919 10507 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:39.690383 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91152.9us
I20260709 12:09:39.708561 11318 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:39.732292 11331 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.200s	user 0.007s	sys 0.000s
W20260709 12:09:39.741688 11340 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
W20260709 12:09:39.789175 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 110809us
I20260709 12:09:39.869880 10430 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:39.903295 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 191304us
I20260709 12:09:39.912729 11221 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.946395 10512 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:39.969642 10506 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:39.966486 10336 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.030598 10335 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:40.037648 10335 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.055648 11219 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.058431 11344 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:40.075578 10584 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.079342 11338 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.080466 10581 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.107437 11351 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:09:40.124727 11221 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.159345 10578 raft_consensus.cc:2955] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.162286 10436 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:40.169538 10437 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:40.180751 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 80de1e7bce9c4d34b0583315cafbc371 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:40.193775 11220 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.196785 11219 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.201126 11348 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:40.203186 11312 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:40.224315 10506 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.236747 11314 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:09:40.241286 10583 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:09:40.241458 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 60232.2us
I20260709 12:09:40.260613 11320 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.263031 11219 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.267158 11312 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.267961 10334 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.276337 11221 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.287060 10581 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:40.293462 11220 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.295693 11343 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:40.296906 11318 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:40.299243 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4642a95b220c4ecea8115de1fa98954e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:40.304375 11197 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.307396 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f5d55f0ab8c4445abbbca950ea76dca1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:40.309296 10504 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:40.313385 11341 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:40.330791 11221 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.331485 10578 raft_consensus.cc:2955] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.336295 10335 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:40.349939 10504 raft_consensus.cc:2955] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.361032 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet 80de1e7bce9c4d34b0583315cafbc371 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:40.376708 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a0bb77704fc9403481da02083315dbb1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:40.380728 10335 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.383473 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 4642a95b220c4ecea8115de1fa98954e with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55454
I20260709 12:09:40.382911 10335 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.400712 10508 raft_consensus.cc:2955] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:40.508843 11353 tablet_replica.cc:333] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:40.519138 11353 raft_consensus.cc:2243] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:40.520292 11353 raft_consensus.cc:2272] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:40.479139 11354 tablet_replica.cc:333] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:09:40.522969 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet a0bb77704fc9403481da02083315dbb1 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:40.537005 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet f5d55f0ab8c4445abbbca950ea76dca1 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:40.573217 11354 raft_consensus.cc:2243] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:40.574347 11354 raft_consensus.cc:2272] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:40.582651 11353 ts_tablet_manager.cc:1916] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:40.595455 11354 ts_tablet_manager.cc:1916] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:40.611488 11355 tablet_replica.cc:333] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:40.625272 11355 raft_consensus.cc:2243] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:40.626286 11355 raft_consensus.cc:2272] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:40.641211 11355 ts_tablet_manager.cc:1916] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:40.647929 11354 ts_tablet_manager.cc:1929] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:40.650434 11354 log.cc:1199] T 4642a95b220c4ecea8115de1fa98954e P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/4642a95b220c4ecea8115de1fa98954e
I20260709 12:09:40.652245 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 4642a95b220c4ecea8115de1fa98954e (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:40.661453 11353 ts_tablet_manager.cc:1929] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:40.662050 11353 log.cc:1199] T 80de1e7bce9c4d34b0583315cafbc371 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/80de1e7bce9c4d34b0583315cafbc371
I20260709 12:09:40.662201 11355 ts_tablet_manager.cc:1929] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:40.664240 11353 tablet_replica.cc:333] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:40.664510 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet 80de1e7bce9c4d34b0583315cafbc371 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:40.665472 11353 raft_consensus.cc:2243] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:40.665604 11355 log.cc:1199] T f5d55f0ab8c4445abbbca950ea76dca1 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/f5d55f0ab8c4445abbbca950ea76dca1
I20260709 12:09:40.666316 11353 raft_consensus.cc:2272] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:40.668339 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet f5d55f0ab8c4445abbbca950ea76dca1 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:40.672437 11353 ts_tablet_manager.cc:1916] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:40.694299 11353 ts_tablet_manager.cc:1929] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:40.694885 11353 log.cc:1199] T a0bb77704fc9403481da02083315dbb1 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/a0bb77704fc9403481da02083315dbb1
I20260709 12:09:40.697707 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet a0bb77704fc9403481da02083315dbb1 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:41.550128 10507 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:41.575016 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66985e55da4c4d3e88491e146eaaded2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:41.579201 10581 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:41.580370 10437 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:41.582113 11313 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:41.584651 11287 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:41.606436 11306 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:41.619949 10581 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:41.621836 10437 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:41.657616 10332 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:41.708505 10437 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:41.739466 10404 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a78d11ad8304de6b3c842374e1a2e09. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:41.802881 10581 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:41.826257 11336 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:41.812901 10506 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:41.864728 11335 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:41.928622 11335 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:41.943212 10581 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:41.954206 10512 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:41.961212 10583 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:41.957504 10332 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:42.013157 10553 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 63843cf9ccd4442cb52e3988597cabd8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:42.016810 10512 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:42.018939 11356 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:42.023517 11368 ts_tablet_manager.cc:933] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:42.034206 10437 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:42.041039 11356 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:42.048903 11356 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.056473 10512 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.084131 10507 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.102056 10332 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:42.112552 11368 tablet_copy_client.cc:323] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:42.115366 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 66985e55da4c4d3e88491e146eaaded2 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:42.116094 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 66985e55da4c4d3e88491e146eaaded2 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-66985e55da4c4d3e88491e146eaaded2
I20260709 12:09:42.130232 10583 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:42.131908 10437 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.134240 10430 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:42.135124 11306 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:09:42.148150 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a082a11e47ba43b2acfaf230875b5e07. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:42.155176 11287 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:42.176978 10523 tablet_copy_source_session.cc:215] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:42.200163 11368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66985e55da4c4d3e88491e146eaaded2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:42.211238 11377 ts_tablet_manager.cc:933] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:42.228895 11368 tablet_copy_client.cc:806] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:42.229765 11368 tablet_copy_client.cc:670] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:42.234006 11368 tablet_copy_client.cc:538] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:42.235647 10578 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.237633 11287 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.245316 10437 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.280910 10333 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:42.300114 11368 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:42.332326 11377 tablet_copy_client.cc:323] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:09:42.334661 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet 9a78d11ad8304de6b3c842374e1a2e09 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:09:42.335290 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet 9a78d11ad8304de6b3c842374e1a2e09 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-9a78d11ad8304de6b3c842374e1a2e09
I20260709 12:09:42.347782 10448 tablet_copy_source_session.cc:215] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:42.351332 11377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78d11ad8304de6b3c842374e1a2e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:42.434561 11377 tablet_copy_client.cc:806] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:42.435364 11377 tablet_copy_client.cc:670] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:42.439435 11368 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:42.440850 11368 tablet_bootstrap.cc:492] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:42.441388 11377 tablet_copy_client.cc:538] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:42.446842 11368 ts_tablet_manager.cc:1403] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.147s	user 0.015s	sys 0.029s
I20260709 12:09:42.452800 11368 raft_consensus.cc:359] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.453832 11368 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:42.454609 11368 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.467518 11368 ts_tablet_manager.cc:1434] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.003s
I20260709 12:09:42.468508 11377 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:42.475000 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-66985e55da4c4d3e88491e146eaaded2 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:42.475497 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-66985e55da4c4d3e88491e146eaaded2 on tablet 66985e55da4c4d3e88491e146eaaded2 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:42.504948 11221 raft_consensus.cc:1217] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:42.546758 11368 ts_tablet_manager.cc:933] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:42.567739 11368 tablet_copy_client.cc:323] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:09:42.569442 11377 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:42.569712 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 63843cf9ccd4442cb52e3988597cabd8 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:09:42.570366 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 63843cf9ccd4442cb52e3988597cabd8 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-63843cf9ccd4442cb52e3988597cabd8
I20260709 12:09:42.570724 11377 tablet_bootstrap.cc:492] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:42.571635 11377 ts_tablet_manager.cc:1403] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.103s	user 0.032s	sys 0.028s
I20260709 12:09:42.575176 11377 raft_consensus.cc:359] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.576174 11377 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:42.576953 11377 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.580229 11377 ts_tablet_manager.cc:1434] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:09:42.581604 10597 tablet_copy_source_session.cc:215] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:42.585564 11368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63843cf9ccd4442cb52e3988597cabd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:42.592203 10448 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-9a78d11ad8304de6b3c842374e1a2e09 received from {username='slave'} at 127.0.0.1:54450
I20260709 12:09:42.592768 10448 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-9a78d11ad8304de6b3c842374e1a2e09 on tablet 9a78d11ad8304de6b3c842374e1a2e09 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:42.631283 11368 tablet_copy_client.cc:806] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:42.632134 11368 tablet_copy_client.cc:670] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:42.636580 11368 tablet_copy_client.cc:538] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:42.649722 11368 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:42.721661 11368 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:42.722756 11368 tablet_bootstrap.cc:492] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:42.723542 11368 ts_tablet_manager.cc:1403] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.019s
I20260709 12:09:42.735543 11368 raft_consensus.cc:359] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.736598 11368 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:42.740320 11368 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.743400 11368 ts_tablet_manager.cc:1434] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:09:42.745226 11221 raft_consensus.cc:1217] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:42.746373 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-63843cf9ccd4442cb52e3988597cabd8 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:09:42.746876 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-63843cf9ccd4442cb52e3988597cabd8 on tablet 63843cf9ccd4442cb52e3988597cabd8 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:42.757938 11377 ts_tablet_manager.cc:933] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:42.771412 11377 tablet_copy_client.cc:323] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:42.773463 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet a082a11e47ba43b2acfaf230875b5e07 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:42.774088 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet a082a11e47ba43b2acfaf230875b5e07 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-a082a11e47ba43b2acfaf230875b5e07
I20260709 12:09:42.782742 10523 tablet_copy_source_session.cc:215] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:42.811625 11377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a082a11e47ba43b2acfaf230875b5e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:42.895156 11377 tablet_copy_client.cc:806] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:42.896064 11377 tablet_copy_client.cc:670] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:42.906633 11377 tablet_copy_client.cc:538] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:42.932885 11377 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:42.985303 11375 raft_consensus.cc:1064] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:42.991526 11377 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:42.993091 11377 tablet_bootstrap.cc:492] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:42.996968 11221 raft_consensus.cc:1217] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:43.011914 11377 ts_tablet_manager.cc:1403] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.079s	user 0.028s	sys 0.012s
I20260709 12:09:43.019533 11377 raft_consensus.cc:359] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:43.020612 11377 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:43.021394 11377 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:42.989327 11375 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:43.032125 11377 ts_tablet_manager.cc:1434] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:09:43.035925 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-a082a11e47ba43b2acfaf230875b5e07 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:43.036559 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-a082a11e47ba43b2acfaf230875b5e07 on tablet a082a11e47ba43b2acfaf230875b5e07 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:43.063041 11221 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.064312 10578 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.064852 10437 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.066207 11287 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:43.067648 11306 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:43.069109 11375 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:43.094281 11375 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.098043 10430 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.122392 11221 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.142300 10583 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.147611 10333 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:43.218291 11220 raft_consensus.cc:1217] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:43.306257 10507 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:43.313609 11221 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:43.316900 10437 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:43.319053 11375 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:43.327227 11306 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:43.334501 11220 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.341374 11392 raft_consensus.cc:1064] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:43.344911 10437 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.330614 11387 raft_consensus.cc:2955] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.380028 10335 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.401890 11392 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:43.439852 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 66985e55da4c4d3e88491e146eaaded2 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55454
I20260709 12:09:43.458787 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 66985e55da4c4d3e88491e146eaaded2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:09:43.553541 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50153.8us
W20260709 12:09:43.555142 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 81295.6us
I20260709 12:09:43.579893 11402 tablet_replica.cc:333] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:09:43.599287 11221 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.614383 11402 raft_consensus.cc:2243] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:43.619216 11397 raft_consensus.cc:1064] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:43.622521 11402 raft_consensus.cc:2272] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:43.623476 11406 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.630398 11397 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:43.632085 11402 ts_tablet_manager.cc:1916] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:43.657737 11306 raft_consensus.cc:1064] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:43.663215 10507 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.665565 10578 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.685885 11335 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.677453 11306 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:09:43.756094 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70793.2us
I20260709 12:09:43.766727 11318 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.788488 11221 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.790064 10578 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.792724 11407 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.801440 11408 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.803687 11402 ts_tablet_manager.cc:1929] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:43.804476 11402 log.cc:1199] T 66985e55da4c4d3e88491e146eaaded2 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/66985e55da4c4d3e88491e146eaaded2
I20260709 12:09:43.806510 10437 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.809345 11408 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.813509 11221 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.820739 11405 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.821533 10504 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:43.825297 11395 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:43.870018 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 66985e55da4c4d3e88491e146eaaded2 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:43.901772 11220 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:43.921250 11415 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
W20260709 12:09:43.933403 11421 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:09:43.876065 11407 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.941083 11404 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.944514 10504 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.952961 11413 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.980154 10437 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.985566 10581 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:43.989914 10430 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:44.003851 10335 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:44.019790 11221 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.030242 10583 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.047412 11219 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.048825 10504 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.068332 10334 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:44.076841 11403 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:44.196051 10430 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.296682 10504 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:44.302163 10332 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:44.310925 10436 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:44.312829 11219 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:44.314901 11287 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:44.316958 11375 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:44.333860 11408 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.357753 10436 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.371734 11219 raft_consensus.cc:2955] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.418058 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a082a11e47ba43b2acfaf230875b5e07 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:44.424780 10430 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:44.479796 10336 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.484162 11219 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:44.502959 10586 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:44.510931 11398 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:44.540262 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet a082a11e47ba43b2acfaf230875b5e07 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55454
I20260709 12:09:44.542513 11430 tablet_replica.cc:333] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:09:44.554734 11430 raft_consensus.cc:2243] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:44.556413 11430 raft_consensus.cc:2272] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:44.561854 11336 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.565894 11221 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.580559 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9a78d11ad8304de6b3c842374e1a2e09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:44.584077 10581 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:44.589028 11399 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:44.587244 11430 ts_tablet_manager.cc:1916] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:44.592715 10335 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:44.634589 11221 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:44.642359 11404 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:44.653475 10430 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:44.663269 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a78d11ad8304de6b3c842374e1a2e09 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:44.657196 11404 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:09:44.663569 10581 raft_consensus.cc:2955] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.676740 11404 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.679194 11219 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.687287 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 63843cf9ccd4442cb52e3988597cabd8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:44.693141 11430 ts_tablet_manager.cc:1929] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:44.694969 11430 log.cc:1199] T a082a11e47ba43b2acfaf230875b5e07 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/a082a11e47ba43b2acfaf230875b5e07
I20260709 12:09:44.698318 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet a082a11e47ba43b2acfaf230875b5e07 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:09:44.711640 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50419.6us
I20260709 12:09:44.726972 10333 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:44.751219 11433 tablet_replica.cc:333] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:44.752669 11433 raft_consensus.cc:2243] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:44.753657 11433 raft_consensus.cc:2272] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:44.777473 10432 raft_consensus.cc:2955] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:44.801047 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet 63843cf9ccd4442cb52e3988597cabd8 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:44.786885 11433 ts_tablet_manager.cc:1916] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:44.860097 11433 ts_tablet_manager.cc:1929] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:44.860647 11433 log.cc:1199] T 9a78d11ad8304de6b3c842374e1a2e09 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/9a78d11ad8304de6b3c842374e1a2e09
I20260709 12:09:44.862983 11433 tablet_replica.cc:333] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:44.893466 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet 9a78d11ad8304de6b3c842374e1a2e09 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:44.923483 11433 raft_consensus.cc:2243] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:44.927174 11433 raft_consensus.cc:2272] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:44.947059 11433 ts_tablet_manager.cc:1916] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:44.993017 11433 ts_tablet_manager.cc:1929] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:44.993886 11433 log.cc:1199] T 63843cf9ccd4442cb52e3988597cabd8 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/63843cf9ccd4442cb52e3988597cabd8
I20260709 12:09:45.006480 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet 63843cf9ccd4442cb52e3988597cabd8 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:09:45.099614 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45426.4us
I20260709 12:09:45.981755 10581 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:46.016484 10553 consensus_peers.cc:597] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f8b08b19bdbb4ac7b6bae9eccebb6335. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:46.020540 10508 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:46.021948 10432 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:46.025321 11431 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:46.026878 11403 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:46.065392 11403 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.073654 10512 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.083267 10432 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.128527 10335 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.131822 10583 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.194434 10508 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:46.194861 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7366ca68fce42f58f5e546ae5cad3ca. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:46.196210 11403 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:46.216157 10437 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:46.229075 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52601.2us
I20260709 12:09:46.233474 11403 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:46.265116 11403 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.266322 10508 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.283192 10583 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.284845 10432 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.311785 10333 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:46.315575 10506 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:46.317257 11403 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:09:46.318212 10553 consensus_peers.cc:597] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bea092b9128d46b6a42f4fe9aa23c739. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:46.320971 10437 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:46.322449 11403 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:46.348381 11404 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.354825 10506 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.372527 10432 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.383925 10508 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.391498 10333 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:46.402879 10583 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:46.405602 10479 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f8b322a475e40fcbca3445b6b0dcf26. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:46.406415 11400 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:46.411378 10436 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:46.414229 11439 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:46.455750 11439 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.459868 10586 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.474242 10436 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:46.492549 10335 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:46.533190 11458 ts_tablet_manager.cc:933] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:46.547752 11458 tablet_copy_client.cc:323] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:09:46.550230 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet f8b08b19bdbb4ac7b6bae9eccebb6335 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:09:46.551049 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet f8b08b19bdbb4ac7b6bae9eccebb6335 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-f8b08b19bdbb4ac7b6bae9eccebb6335
I20260709 12:09:46.563428 10597 tablet_copy_source_session.cc:215] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:46.568079 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b08b19bdbb4ac7b6bae9eccebb6335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:46.603091 11458 tablet_copy_client.cc:806] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:46.603884 11458 tablet_copy_client.cc:670] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:46.609032 11458 tablet_copy_client.cc:538] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:46.624987 11458 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:46.654863 11458 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:46.655980 11458 tablet_bootstrap.cc:492] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:46.658892 11458 ts_tablet_manager.cc:1403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.034s	user 0.023s	sys 0.004s
I20260709 12:09:46.666869 11458 raft_consensus.cc:359] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.667917 11458 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:46.668882 11458 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.679848 11458 ts_tablet_manager.cc:1434] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.004s
I20260709 12:09:46.688509 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-f8b08b19bdbb4ac7b6bae9eccebb6335 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:09:46.689129 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-f8b08b19bdbb4ac7b6bae9eccebb6335 on tablet f8b08b19bdbb4ac7b6bae9eccebb6335 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:46.762697 11458 ts_tablet_manager.cc:933] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:46.785419 11458 tablet_copy_client.cc:323] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:09:46.787190 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet c7366ca68fce42f58f5e546ae5cad3ca from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:09:46.787788 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet c7366ca68fce42f58f5e546ae5cad3ca from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-c7366ca68fce42f58f5e546ae5cad3ca
I20260709 12:09:46.796586 10597 tablet_copy_source_session.cc:215] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:46.802001 11464 ts_tablet_manager.cc:933] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:46.806681 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7366ca68fce42f58f5e546ae5cad3ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:46.829151 11464 tablet_copy_client.cc:323] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:09:46.831468 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet bea092b9128d46b6a42f4fe9aa23c739 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:09:46.832062 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet bea092b9128d46b6a42f4fe9aa23c739 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-bea092b9128d46b6a42f4fe9aa23c739
I20260709 12:09:46.844512 10597 tablet_copy_source_session.cc:215] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:46.831699 11458 tablet_copy_client.cc:806] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:46.847368 11458 tablet_copy_client.cc:670] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:46.848588 11464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bea092b9128d46b6a42f4fe9aa23c739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:46.851641 11458 tablet_copy_client.cc:538] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:46.871443 11458 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:46.893116 11464 tablet_copy_client.cc:806] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:46.893916 11464 tablet_copy_client.cc:670] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:46.910878 11458 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:46.913394 11458 tablet_bootstrap.cc:492] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:46.914176 11458 ts_tablet_manager.cc:1403] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.009s
I20260709 12:09:46.917547 11458 raft_consensus.cc:359] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.918438 11458 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:46.919113 11458 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:46.926095 11464 tablet_copy_client.cc:538] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:46.929229 11458 ts_tablet_manager.cc:1434] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.004s
I20260709 12:09:46.931133 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-c7366ca68fce42f58f5e546ae5cad3ca received from {username='slave'} at 127.0.0.1:36746
I20260709 12:09:46.931577 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-c7366ca68fce42f58f5e546ae5cad3ca on tablet c7366ca68fce42f58f5e546ae5cad3ca with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:46.939829 11464 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:47.032045 11458 ts_tablet_manager.cc:933] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:47.105475 11458 tablet_copy_client.cc:323] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:47.112201 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 9f8b322a475e40fcbca3445b6b0dcf26 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:47.112969 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 9f8b322a475e40fcbca3445b6b0dcf26 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-9f8b322a475e40fcbca3445b6b0dcf26
I20260709 12:09:47.124778 10523 tablet_copy_source_session.cc:215] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:47.166225 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8b322a475e40fcbca3445b6b0dcf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:47.184774 11221 raft_consensus.cc:1217] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:09:47.215546 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60557.6us
I20260709 12:09:47.233263 11458 tablet_copy_client.cc:806] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:47.234139 11458 tablet_copy_client.cc:670] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:47.238667 11458 tablet_copy_client.cc:538] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:47.258009 11464 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:47.259385 11464 tablet_bootstrap.cc:492] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:47.261550 11464 ts_tablet_manager.cc:1403] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.322s	user 0.022s	sys 0.008s
I20260709 12:09:47.265002 11458 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:47.269629 11464 raft_consensus.cc:359] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:47.275388 11464 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:47.276455 11464 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:47.318851 11219 raft_consensus.cc:1217] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:47.357241 11458 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:47.358441 11458 tablet_bootstrap.cc:492] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:47.359308 11458 ts_tablet_manager.cc:1403] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.095s	user 0.035s	sys 0.023s
I20260709 12:09:47.370546 11464 ts_tablet_manager.cc:1434] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.000s
I20260709 12:09:47.375672 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-bea092b9128d46b6a42f4fe9aa23c739 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:09:47.376242 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-bea092b9128d46b6a42f4fe9aa23c739 on tablet bea092b9128d46b6a42f4fe9aa23c739 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:47.373915 11458 raft_consensus.cc:359] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:47.378770 11458 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:47.379552 11458 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:47.382534 11458 ts_tablet_manager.cc:1434] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:09:47.384443 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-9f8b322a475e40fcbca3445b6b0dcf26 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:47.384910 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-9f8b322a475e40fcbca3445b6b0dcf26 on tablet 9f8b322a475e40fcbca3445b6b0dcf26 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:47.397781 11221 raft_consensus.cc:1217] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:47.408695 11221 raft_consensus.cc:1217] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:47.613808 11470 raft_consensus.cc:1064] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:47.616936 11470 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:47.639832 11221 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:47.642027 11468 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:47.659639 10508 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:47.667178 11470 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:47.675166 10432 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:47.682516 11431 raft_consensus.cc:1064] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:47.685485 11431 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:47.693711 11404 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:47.706146 11404 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.726560 11221 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.730912 10507 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.790314 10335 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:47.875361 10436 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.877255 11221 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:47.882591 10437 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:47.882478 11431 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:47.885460 11468 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:47.886771 10507 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:47.889328 11470 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:47.912561 10586 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:47.931857 11219 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:47.932783 10506 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:47.933753 11468 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:47.935688 11468 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:47.949834 10506 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.950263 11468 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.959403 11219 raft_consensus.cc:2955] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.973886 10333 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.999665 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f8b08b19bdbb4ac7b6bae9eccebb6335 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:48.000451 10507 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:47.994273 11404 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.036743 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet f8b08b19bdbb4ac7b6bae9eccebb6335 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:48.051644 11472 raft_consensus.cc:1064] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:48.056286 11472 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:48.060221 10333 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:48.065896 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51472.9us
I20260709 12:09:48.069937 10436 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.081193 11431 raft_consensus.cc:1064] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:48.083958 11221 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:48.085837 11462 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:48.090351 11431 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:48.098551 10584 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:48.100047 11462 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:48.100425 10432 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:48.105684 11462 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:48.142419 11473 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.144133 11492 tablet_replica.cc:333] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:48.150043 10432 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.156613 10581 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.169580 11221 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.185485 11492 raft_consensus.cc:2243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:48.186515 11492 raft_consensus.cc:2272] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:48.190485 11492 ts_tablet_manager.cc:1916] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:48.205552 10333 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.215879 10507 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:48.217759 11491 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:48.227571 11219 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.259143 10584 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:09:48.266783 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53384.5us
I20260709 12:09:48.268818 11492 ts_tablet_manager.cc:1929] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:48.269644 11492 log.cc:1199] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/f8b08b19bdbb4ac7b6bae9eccebb6335
I20260709 12:09:48.269793 11220 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:48.272225 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet f8b08b19bdbb4ac7b6bae9eccebb6335 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:48.274972 10504 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:48.294770 11218 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:48.301340 11220 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:48.308506 11473 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:48.321132 11473 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.325178 11431 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:48.325995 11220 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.332834 11431 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:48.361661 11490 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:48.377355 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45438.7us
I20260709 12:09:48.380677 11218 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.398373 10438 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:48.396642 10431 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:48.400231 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c7366ca68fce42f58f5e546ae5cad3ca with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:48.405308 10437 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:48.411453 11462 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:48.417959 10437 raft_consensus.cc:2955] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.426625 10333 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.439903 11490 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.445642 11219 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.448390 10332 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:48.465054 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f8b322a475e40fcbca3445b6b0dcf26 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:48.491446 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f8b322a475e40fcbca3445b6b0dcf26 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55454
I20260709 12:09:48.518532 10504 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.526373 11504 tablet_replica.cc:333] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:09:48.528041 11431 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:48.529999 11504 raft_consensus.cc:2243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:48.531311 11504 raft_consensus.cc:2272] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:48.531221 11470 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:48.545743 11504 ts_tablet_manager.cc:1916] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:48.577549 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet c7366ca68fce42f58f5e546ae5cad3ca with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:48.587029 10336 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.628646 10438 raft_consensus.cc:2955] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.649370 11506 tablet_replica.cc:333] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:48.662789 10578 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:48.669358 11219 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:48.671823 11431 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:48.673630 11506 raft_consensus.cc:2243] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:48.683357 10432 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.686980 10508 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:48.689028 11470 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:48.693948 11490 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.675081 11506 raft_consensus.cc:2272] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:48.713004 10508 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.719204 11506 ts_tablet_manager.cc:1916] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:48.709209 11219 raft_consensus.cc:2955] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.736268 11504 ts_tablet_manager.cc:1929] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:48.737025 11504 log.cc:1199] T 9f8b322a475e40fcbca3445b6b0dcf26 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/9f8b322a475e40fcbca3445b6b0dcf26
I20260709 12:09:48.751559 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 9f8b322a475e40fcbca3445b6b0dcf26 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:48.764806 11506 ts_tablet_manager.cc:1929] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:48.765537 11506 log.cc:1199] T c7366ca68fce42f58f5e546ae5cad3ca P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/c7366ca68fce42f58f5e546ae5cad3ca
I20260709 12:09:48.768913 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet c7366ca68fce42f58f5e546ae5cad3ca (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:48.771306 10333 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:48.781128 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bea092b9128d46b6a42f4fe9aa23c739 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:48.801815 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet bea092b9128d46b6a42f4fe9aa23c739 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:48.814821 11508 tablet_replica.cc:333] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:48.816627 11508 raft_consensus.cc:2243] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:48.817551 11508 raft_consensus.cc:2272] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:48.822048 11508 ts_tablet_manager.cc:1916] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:48.846658 11508 ts_tablet_manager.cc:1929] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:48.847296 11508 log.cc:1199] T bea092b9128d46b6a42f4fe9aa23c739 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/bea092b9128d46b6a42f4fe9aa23c739
I20260709 12:09:48.849769 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet bea092b9128d46b6a42f4fe9aa23c739 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:49.878037 10583 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:49.885910 10553 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4689962670ee421e8993be8a9ba7a657. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:49.886739 10507 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:49.888365 10432 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:49.888414 11431 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:49.890534 11431 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:49.897840 11490 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:49.899888 10507 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:49.902544 10432 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:49.918954 10507 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:49.920744 10336 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:49.933491 10432 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:49.936019 10479 consensus_peers.cc:597] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38ead18e1a814c8b958f80396faa3170. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:49.937144 11439 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:49.937999 10583 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:49.942864 11472 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:49.994974 11472 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.001068 10578 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.032320 10507 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.034281 10432 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.065407 10583 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:50.067183 11514 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:50.067782 10333 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:50.072554 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3872158fef542f0b0f902e3a8756feb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:50.075625 10432 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:50.078179 11472 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:50.085083 11514 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.087432 10578 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.112210 10432 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.113467 10438 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.120289 10335 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:50.147703 10404 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60f72c0bed7c45e484efbed50d2c364a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:50.150079 10586 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:50.151525 10507 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:50.153812 11336 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:50.158414 11518 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:50.168061 11518 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.169907 10507 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.196223 10586 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:50.199437 10333 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:50.337955 11528 ts_tablet_manager.cc:933] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:50.340566 11528 tablet_copy_client.cc:323] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:50.342392 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 38ead18e1a814c8b958f80396faa3170 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:50.342926 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 38ead18e1a814c8b958f80396faa3170 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-38ead18e1a814c8b958f80396faa3170
I20260709 12:09:50.349133 10523 tablet_copy_source_session.cc:215] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:50.352317 11528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ead18e1a814c8b958f80396faa3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:50.367520 11528 tablet_copy_client.cc:806] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:50.368320 11528 tablet_copy_client.cc:670] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:50.373641 11528 tablet_copy_client.cc:538] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:50.385181 11528 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:50.410562 11528 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:50.411751 11528 tablet_bootstrap.cc:492] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:50.412572 11528 ts_tablet_manager.cc:1403] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.005s
I20260709 12:09:50.416198 11528 raft_consensus.cc:359] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.417160 11528 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:50.417968 11528 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.422120 11528 ts_tablet_manager.cc:1434] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:09:50.424678 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-38ead18e1a814c8b958f80396faa3170 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:50.425175 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-38ead18e1a814c8b958f80396faa3170 on tablet 38ead18e1a814c8b958f80396faa3170 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:50.429543 11528 ts_tablet_manager.cc:933] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:50.434155 11528 tablet_copy_client.cc:323] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:09:50.436088 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 4689962670ee421e8993be8a9ba7a657 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:09:50.436717 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 4689962670ee421e8993be8a9ba7a657 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-4689962670ee421e8993be8a9ba7a657
I20260709 12:09:50.447942 10597 tablet_copy_source_session.cc:215] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:50.451221 11528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4689962670ee421e8993be8a9ba7a657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:50.469089 11528 tablet_copy_client.cc:806] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:50.469769 11528 tablet_copy_client.cc:670] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:50.474893 11528 tablet_copy_client.cc:538] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:50.483094 11528 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:50.503429 11528 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:50.504350 11528 tablet_bootstrap.cc:492] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:50.505309 11528 ts_tablet_manager.cc:1403] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.008s
I20260709 12:09:50.509013 11528 raft_consensus.cc:359] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.510051 11528 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:50.510960 11528 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.513861 11528 ts_tablet_manager.cc:1434] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:09:50.520043 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4689962670ee421e8993be8a9ba7a657 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:09:50.520515 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4689962670ee421e8993be8a9ba7a657 on tablet 4689962670ee421e8993be8a9ba7a657 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:50.563503 11528 ts_tablet_manager.cc:933] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:50.570451 11532 ts_tablet_manager.cc:933] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:50.572338 11528 tablet_copy_client.cc:323] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:50.573741 11532 tablet_copy_client.cc:323] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:09:50.574185 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet b3872158fef542f0b0f902e3a8756feb from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:50.574723 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet b3872158fef542f0b0f902e3a8756feb from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-b3872158fef542f0b0f902e3a8756feb
I20260709 12:09:50.577277 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet 60f72c0bed7c45e484efbed50d2c364a from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:09:50.577921 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet 60f72c0bed7c45e484efbed50d2c364a from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-60f72c0bed7c45e484efbed50d2c364a
I20260709 12:09:50.583626 10523 tablet_copy_source_session.cc:215] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:50.587371 11528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3872158fef542f0b0f902e3a8756feb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:50.587683 10448 tablet_copy_source_session.cc:215] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:50.592655 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f72c0bed7c45e484efbed50d2c364a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:50.605278 11528 tablet_copy_client.cc:806] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:50.605938 11528 tablet_copy_client.cc:670] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:50.614296 11528 tablet_copy_client.cc:538] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:50.614717 11532 tablet_copy_client.cc:806] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:50.615383 11532 tablet_copy_client.cc:670] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:50.624154 11528 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:50.633162 11532 tablet_copy_client.cc:538] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:50.653007 11528 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:50.654119 11528 tablet_bootstrap.cc:492] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:50.654915 11528 ts_tablet_manager.cc:1403] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.008s
I20260709 12:09:50.657891 11528 raft_consensus.cc:359] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.658679 11528 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:50.659309 11528 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.666257 11528 ts_tablet_manager.cc:1434] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:09:50.668645 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-b3872158fef542f0b0f902e3a8756feb received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:50.669085 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-b3872158fef542f0b0f902e3a8756feb on tablet b3872158fef542f0b0f902e3a8756feb with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:50.686719 11532 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:50.726959 11532 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:50.728081 11532 tablet_bootstrap.cc:492] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:50.728902 11532 ts_tablet_manager.cc:1403] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.042s	user 0.027s	sys 0.005s
I20260709 12:09:50.732342 11532 raft_consensus.cc:359] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.733283 11532 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:50.734004 11532 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:50.742268 11532 ts_tablet_manager.cc:1434] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:09:50.750119 10448 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-60f72c0bed7c45e484efbed50d2c364a received from {username='slave'} at 127.0.0.1:54450
I20260709 12:09:50.750696 10448 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-60f72c0bed7c45e484efbed50d2c364a on tablet 60f72c0bed7c45e484efbed50d2c364a with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:50.803110 11218 raft_consensus.cc:1217] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:50.986604 11219 raft_consensus.cc:1217] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:51.099234 11218 raft_consensus.cc:1217] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:51.129097 11218 raft_consensus.cc:1217] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:51.192800 11543 raft_consensus.cc:1064] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:51.195218 11543 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.202028 10578 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.202453 11218 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.203383 10432 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.203761 11473 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:51.204845 11472 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:51.205886 11543 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.219918 11514 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.223793 11218 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.226130 10578 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.241498 10336 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.254207 10432 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.300078 10504 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.308063 11218 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.310204 11514 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:51.310858 10578 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.312860 11472 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:51.320696 11473 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.322603 10578 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.328517 11218 raft_consensus.cc:2955] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.337764 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 38ead18e1a814c8b958f80396faa3170 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:51.341574 10333 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.354399 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet 38ead18e1a814c8b958f80396faa3170 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:51.356905 11552 tablet_replica.cc:333] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:51.358079 11552 raft_consensus.cc:2243] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:51.358858 11552 raft_consensus.cc:2272] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:51.362229 11552 ts_tablet_manager.cc:1916] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:51.377562 11552 ts_tablet_manager.cc:1929] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:51.378016 11552 log.cc:1199] T 38ead18e1a814c8b958f80396faa3170 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/38ead18e1a814c8b958f80396faa3170
I20260709 12:09:51.379784 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet 38ead18e1a814c8b958f80396faa3170 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:51.438097 11542 raft_consensus.cc:1064] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:51.440689 11542 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.452471 11218 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.454854 10504 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.454720 11431 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.455994 10432 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.457885 11431 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:51.459900 11431 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:51.495635 11431 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.498854 10507 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.499185 11218 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.514607 10437 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.521507 10335 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.548210 10578 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.600226 11218 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.603394 11542 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:51.611912 11431 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.617636 10437 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.624825 11473 raft_consensus.cc:1064] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:51.626710 11554 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.627722 11473 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.643047 11219 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.662349 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4689962670ee421e8993be8a9ba7a657 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:51.668568 10437 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.670740 11543 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.672895 11220 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.676409 10578 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.677047 11543 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.690953 11543 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:09:51.693589 10335 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.713562 11336 raft_consensus.cc:1064] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:51.715817 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet 4689962670ee421e8993be8a9ba7a657 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:51.715692 10437 raft_consensus.cc:2955] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.728142 11336 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.733376 11563 tablet_replica.cc:333] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:51.738878 11563 raft_consensus.cc:2243] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:51.738261 11514 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.748878 11563 raft_consensus.cc:2272] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:51.765480 11218 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.768268 10583 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.800874 10336 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.807416 11563 ts_tablet_manager.cc:1916] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:51.832410 11218 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.840164 11538 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.845551 10432 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.874220 10583 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:51.878439 10507 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:51.887101 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84749.9us
I20260709 12:09:51.895398 11538 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.905439 10507 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:51.912076 11559 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:51.914126 11218 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.916070 11472 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.924448 10432 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.926231 11472 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.929963 11567 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.951198 11219 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.969098 11559 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.977718 10437 raft_consensus.cc:2955] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.990926 11563 ts_tablet_manager.cc:1929] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:51.991844 11563 log.cc:1199] T 4689962670ee421e8993be8a9ba7a657 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/4689962670ee421e8993be8a9ba7a657
I20260709 12:09:51.996110 10335 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:51.999576 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet 4689962670ee421e8993be8a9ba7a657 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:52.009150 10584 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.035182 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b3872158fef542f0b0f902e3a8756feb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:52.038769 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet b3872158fef542f0b0f902e3a8756feb with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55454
I20260709 12:09:52.052057 10332 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.054504 11572 tablet_replica.cc:333] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:09:52.081427 11219 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.085309 11572 raft_consensus.cc:2243] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:52.086742 11572 raft_consensus.cc:2272] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:52.105288 11572 ts_tablet_manager.cc:1916] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:52.108506 10507 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.114534 10437 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:52.125734 11572 ts_tablet_manager.cc:1929] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:52.126420 11572 log.cc:1199] T b3872158fef542f0b0f902e3a8756feb P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/b3872158fef542f0b0f902e3a8756feb
I20260709 12:09:52.130570 11219 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:52.132241 11559 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:52.134454 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet b3872158fef542f0b0f902e3a8756feb (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:52.161336 11218 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.158084 11571 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.172843 10584 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:52.182695 11500 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:52.189142 10584 raft_consensus.cc:2955] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.193375 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 60f72c0bed7c45e484efbed50d2c364a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:52.196305 10332 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:52.209847 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet 60f72c0bed7c45e484efbed50d2c364a with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:52.211182 11563 tablet_replica.cc:333] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:52.212415 11563 raft_consensus.cc:2243] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:52.213344 11563 raft_consensus.cc:2272] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:52.225445 11563 ts_tablet_manager.cc:1916] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:52.250300 11563 ts_tablet_manager.cc:1929] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:52.251040 11563 log.cc:1199] T 60f72c0bed7c45e484efbed50d2c364a P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/60f72c0bed7c45e484efbed50d2c364a
I20260709 12:09:52.252794 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet 60f72c0bed7c45e484efbed50d2c364a (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:53.290776 10504 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:53.303803 10479 consensus_peers.cc:597] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06d2724d97aa43298ebaf9e746580fd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.304087 10438 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.306249 10578 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.306282 11514 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.308590 11575 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:53.319085 11567 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.320780 10438 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.321987 10578 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.350414 10438 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:53.351118 10332 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:09:53.368412 10404 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51452865e973445d9e13f232cfb5efb9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.369012 10583 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:53.370661 10504 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:53.373020 11583 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.374835 11573 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:09:53.397790 11580 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.404424 10504 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.403887 10583 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.438649 10504 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:53.455921 10335 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.458463 10578 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:53.456408 10479 consensus_peers.cc:597] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9042bec741aa476587972e688997da43. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.467244 10438 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.469154 11575 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:53.471190 11567 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:53.503163 11514 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.506795 10578 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.532809 10336 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.538280 10437 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:53.553376 10438 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.557922 10504 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.558104 10578 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:53.560403 10404 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4d2e9ee3d824817989d1fa5abcd9d67. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.569591 11574 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:09:53.580840 11580 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.594787 11574 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.602259 10578 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.608428 10504 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.629613 10335 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:53.851334 11596 ts_tablet_manager.cc:933] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:53.879014 11596 tablet_copy_client.cc:323] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:09:53.881533 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet 51452865e973445d9e13f232cfb5efb9 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:09:53.882169 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet 51452865e973445d9e13f232cfb5efb9 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-51452865e973445d9e13f232cfb5efb9
I20260709 12:09:53.885274 11599 ts_tablet_manager.cc:933] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:53.891250 10448 tablet_copy_source_session.cc:215] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:53.895740 11596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51452865e973445d9e13f232cfb5efb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:53.900154 11599 tablet_copy_client.cc:323] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:53.902092 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 06d2724d97aa43298ebaf9e746580fd0 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:53.902657 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 06d2724d97aa43298ebaf9e746580fd0 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-06d2724d97aa43298ebaf9e746580fd0
I20260709 12:09:53.905407 11598 ts_tablet_manager.cc:933] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:53.913447 10523 tablet_copy_source_session.cc:215] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:53.925298 11598 tablet_copy_client.cc:323] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:09:53.927886 11596 tablet_copy_client.cc:806] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:53.928784 11596 tablet_copy_client.cc:670] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:53.925299 11599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d2724d97aa43298ebaf9e746580fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:53.931229 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 9042bec741aa476587972e688997da43 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:09:53.931849 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 9042bec741aa476587972e688997da43 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-9042bec741aa476587972e688997da43
I20260709 12:09:53.934094 11596 tablet_copy_client.cc:538] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:53.940382 10523 tablet_copy_source_session.cc:215] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:53.944046 11598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9042bec741aa476587972e688997da43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:53.945727 11596 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:53.961086 11599 tablet_copy_client.cc:806] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:53.965386 11598 tablet_copy_client.cc:806] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:53.966084 11599 tablet_copy_client.cc:670] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:53.967187 11598 tablet_copy_client.cc:670] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:53.971038 11599 tablet_copy_client.cc:538] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:53.971987 11598 tablet_copy_client.cc:538] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:53.982468 11598 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:53.993084 11599 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:53.999246 11596 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.000425 11596 tablet_bootstrap.cc:492] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:54.001267 11596 ts_tablet_manager.cc:1403] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.010s
I20260709 12:09:54.005034 11596 raft_consensus.cc:359] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.006129 11596 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:54.016775 11596 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.021703 11598 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.022893 11598 tablet_bootstrap.cc:492] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:54.023780 11598 ts_tablet_manager.cc:1403] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.016s
I20260709 12:09:54.027287 11598 raft_consensus.cc:359] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.028249 11598 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:54.028971 11598 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.032405 11596 ts_tablet_manager.cc:1434] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.031s	user 0.012s	sys 0.002s
I20260709 12:09:54.038725 11598 ts_tablet_manager.cc:1434] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.007s
I20260709 12:09:54.042459 10448 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-51452865e973445d9e13f232cfb5efb9 received from {username='slave'} at 127.0.0.1:54450
I20260709 12:09:54.043054 10448 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-51452865e973445d9e13f232cfb5efb9 on tablet 51452865e973445d9e13f232cfb5efb9 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:54.044999 11599 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.046155 11599 tablet_bootstrap.cc:492] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:54.047904 10522 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-9042bec741aa476587972e688997da43 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:54.048389 10522 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-9042bec741aa476587972e688997da43 on tablet 9042bec741aa476587972e688997da43 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:54.049757 11599 ts_tablet_manager.cc:1403] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.057s	user 0.022s	sys 0.009s
I20260709 12:09:54.054054 11596 ts_tablet_manager.cc:933] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:09:54.056108 11599 raft_consensus.cc:359] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.059957 11599 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:54.060715 11599 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.069625 11599 ts_tablet_manager.cc:1434] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:09:54.072907 10522 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-06d2724d97aa43298ebaf9e746580fd0 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:09:54.073560 10522 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-06d2724d97aa43298ebaf9e746580fd0 on tablet 06d2724d97aa43298ebaf9e746580fd0 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:54.081372 11596 tablet_copy_client.cc:323] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:09:54.083304 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet b4d2e9ee3d824817989d1fa5abcd9d67 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:09:54.083878 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet b4d2e9ee3d824817989d1fa5abcd9d67 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-b4d2e9ee3d824817989d1fa5abcd9d67
I20260709 12:09:54.094282 10448 tablet_copy_source_session.cc:215] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:54.098001 11596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4d2e9ee3d824817989d1fa5abcd9d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:54.115469 11596 tablet_copy_client.cc:806] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:54.116127 11596 tablet_copy_client.cc:670] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:54.122612 11596 tablet_copy_client.cc:538] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:54.136144 11596 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:09:54.158248 11596 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.159214 11596 tablet_bootstrap.cc:492] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:09:54.160118 11596 ts_tablet_manager.cc:1403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.008s
I20260709 12:09:54.163219 11596 raft_consensus.cc:359] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.163954 11596 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:09:54.164676 11596 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:54.166970 11596 ts_tablet_manager.cc:1434] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:09:54.169195 10448 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-b4d2e9ee3d824817989d1fa5abcd9d67 received from {username='slave'} at 127.0.0.1:54450
I20260709 12:09:54.169683 10448 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-b4d2e9ee3d824817989d1fa5abcd9d67 on tablet b4d2e9ee3d824817989d1fa5abcd9d67 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:54.304001 11219 raft_consensus.cc:1217] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:09:54.459003 11218 raft_consensus.cc:1217] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:54.544253 11218 raft_consensus.cc:1217] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:54.561179 11218 raft_consensus.cc:1217] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:54.761085 11610 raft_consensus.cc:1064] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:54.764140 11610 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:54.772116 11218 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:09:54.772410 10578 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:09:54.773556 10508 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:09:54.775481 11615 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:54.777311 11580 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:54.778699 11615 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:54.799122 11615 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.800542 10578 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.810089 10508 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.815230 11219 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.821841 10332 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:54.848192 11592 raft_consensus.cc:1064] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:54.859292 11592 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:54.873705 11219 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:54.875099 10438 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:54.876555 10578 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:54.878474 11567 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:54.894610 11618 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:54.896873 11618 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:54.911021 10432 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:54.923152 11219 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:09:54.923899 10508 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:09:54.925156 11613 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:54.926313 11610 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:54.929196 11567 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.941043 11219 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.953439 11610 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.983076 10333 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:54.990269 11618 raft_consensus.cc:1064] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:54.993299 11618 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:55.013496 11220 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.013386 10504 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.017983 11618 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:55.018363 11218 raft_consensus.cc:2955] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.032299 11580 raft_consensus.cc:1064] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:09:55.035593 11580 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:09:55.072521 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44537.3us
I20260709 12:09:55.078472 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51452865e973445d9e13f232cfb5efb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:55.098093 10578 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.097317 10335 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.109229 11218 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.116694 10438 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.118896 10504 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.120798 10432 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.142702 10584 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.141742 10578 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:09:55.168812 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 85577.2us
I20260709 12:09:55.192847 11592 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.195042 11618 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.207819 10508 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:55.239751 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 51452865e973445d9e13f232cfb5efb9 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55454
I20260709 12:09:55.216718 11637 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 12:09:55.213418 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 89429.8us
I20260709 12:09:55.283437 11645 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.306289 11218 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.260950 11648 tablet_replica.cc:333] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
W20260709 12:09:55.352596 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46175.5us
W20260709 12:09:55.366479 11649 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 12:09:55.372082 10587 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.381539 11650 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.390400 11628 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:55.397872 10584 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.406418 11634 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.409777 11642 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.418272 11592 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:55.427480 10335 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.434190 11648 raft_consensus.cc:2243] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:55.435250 11648 raft_consensus.cc:2272] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:55.436822 10584 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.446540 11219 raft_consensus.cc:2955] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.465257 11648 ts_tablet_manager.cc:1916] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:55.485916 10438 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.486616 11218 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.507673 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 06d2724d97aa43298ebaf9e746580fd0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:55.512269 10335 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.534476 10504 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:55.517833 11648 ts_tablet_manager.cc:1929] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:09:55.542670 11648 log.cc:1199] T 51452865e973445d9e13f232cfb5efb9 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/51452865e973445d9e13f232cfb5efb9
I20260709 12:09:55.542742 10583 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.543190 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet 06d2724d97aa43298ebaf9e746580fd0 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:55.545075 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 51452865e973445d9e13f232cfb5efb9 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:55.546952 11592 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:55.563567 11219 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.570495 11628 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.575369 11655 tablet_replica.cc:333] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:55.578652 11650 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:55.591126 11655 raft_consensus.cc:2243] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:55.590457 11218 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.595760 11642 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.590703 10584 raft_consensus.cc:2955] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.592361 11655 raft_consensus.cc:2272] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:55.605234 11655 ts_tablet_manager.cc:1916] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:55.616046 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9042bec741aa476587972e688997da43 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:55.617188 11219 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.635330 10335 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 3 to 4, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:55.635495 11655 ts_tablet_manager.cc:1929] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:55.641330 11655 log.cc:1199] T 06d2724d97aa43298ebaf9e746580fd0 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/06d2724d97aa43298ebaf9e746580fd0
I20260709 12:09:55.643551 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet 06d2724d97aa43298ebaf9e746580fd0 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:55.670482 10587 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.681825 10336 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:55.658778 10507 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.715699 10430 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:55.725628 11219 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.725950 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet 9042bec741aa476587972e688997da43 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48980
I20260709 12:09:55.727350 11645 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:55.731285 11655 tablet_replica.cc:333] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:09:55.732610 11655 raft_consensus.cc:2243] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:55.733595 11655 raft_consensus.cc:2272] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:55.742530 11655 ts_tablet_manager.cc:1916] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:55.742870 11615 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.749320 11219 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.755409 10587 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.757963 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b4d2e9ee3d824817989d1fa5abcd9d67 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:55.768916 10336 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:55.778910 11642 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:55.782752 11655 ts_tablet_manager.cc:1929] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:55.783406 11655 log.cc:1199] T 9042bec741aa476587972e688997da43 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/9042bec741aa476587972e688997da43
I20260709 12:09:55.785715 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet 9042bec741aa476587972e688997da43 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:55.789284 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet b4d2e9ee3d824817989d1fa5abcd9d67 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41062
I20260709 12:09:55.800365 10587 raft_consensus.cc:2955] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:09:55.805124 11656 tablet_replica.cc:333] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:09:55.805434  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:55.806644 11656 raft_consensus.cc:2243] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:55.807593 11656 raft_consensus.cc:2272] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:55.812695 11656 ts_tablet_manager.cc:1916] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:55.831029 11656 ts_tablet_manager.cc:1929] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:55.831575 11656 log.cc:1199] T b4d2e9ee3d824817989d1fa5abcd9d67 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/b4d2e9ee3d824817989d1fa5abcd9d67
I20260709 12:09:55.835390 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet b4d2e9ee3d824817989d1fa5abcd9d67 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:09:55.890341 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:09:55.891171 10507 raft_consensus.cc:606] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:09:55.894304 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:09:55.894799 10507 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:09:55.908667 10430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:09:55.909226 10430 raft_consensus.cc:606] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:55.911774 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:09:55.912226 10507 raft_consensus.cc:606] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:55.914366 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:09:55.914760 10507 raft_consensus.cc:606] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:09:55.915956  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:09:55.917657  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:55.918972  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:56.030649 11643 raft_consensus.cc:993] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:09:56.031744 11650 raft_consensus.cc:1081] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:09:56.033677 11219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57904
I20260709 12:09:56.034380 11219 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:56.034758 11219 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.041656 11219 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:56.044057 11219 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:56.044836 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:56.045341 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:56.045691 10430 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.046041 10506 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:56.046404 10506 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:56.047143 10506 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:56.048442 10506 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.052640 10430 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:09:56.053854 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:56.054720 11606 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.055497 10506 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:09:56.056438 11606 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:09:56.057494 11606 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:56.067566 10336 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:56.173834 11650 raft_consensus.cc:993] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:09:56.174472 11643 raft_consensus.cc:1081] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:09:56.176656 10579 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:09:56.177284 10579 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:56.177690 10579 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.185982 10579 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.188351 10579 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:56.189016 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:56.189287 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:56.189730 10430 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.189836 10506 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:56.190217 10506 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:56.191035 10506 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.192334 10506 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.200245 10506 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:09:56.201215 10430 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:09:56.201514 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:56.202731 11617 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.203236 11617 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:56.204217 11617 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.216660 10336 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:56.243202 11592 raft_consensus.cc:993] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:09:56.243793 11643 raft_consensus.cc:1081] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:09:56.245340 10584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:09:56.245862 10584 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:56.246145 10584 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.254029 10584 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.255575 10584 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:56.256196 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:56.256479 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:56.256857 10430 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.257133 10506 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:56.257511 10506 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:56.258285 10506 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.259655 10506 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.263294 10430 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:09:56.264526 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:56.265412 11617 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.266260 11617 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:56.267030 10506 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:09:56.267375 11617 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.283155 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:56.311997 11592 raft_consensus.cc:993] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:09:56.312546 11643 raft_consensus.cc:1081] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:09:56.313977 10587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:09:56.314473 10587 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:56.314785 10587 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.321527 10587 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.323132 10587 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:56.323994 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:56.324725 10430 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.324767 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:09:56.325563 10507 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:56.326241 10507 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:56.327122 10507 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.328451 10507 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.333051 10430 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:09:56.334300 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:56.335165 11617 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.335665 11617 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:56.336374 10507 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:09:56.337205 11617 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.348794 10336 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:56.399747 11642 raft_consensus.cc:993] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:09:56.400394 11636 raft_consensus.cc:1081] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:09:56.402004 10506 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:09:56.402613 10506 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:56.403023 10506 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.410388 10506 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.412211 10506 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:56.412902 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:56.413318 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:56.413543 10430 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:56.413877 10430 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:09:56.413903 10587 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.414597 10430 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.415658 10430 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.420374 10587 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:09:56.421543 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:09:56.422310 10430 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:09:56.422405 11643 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.423012 11643 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:56.423945 11643 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:56.436981 10336 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:56.527585 10430 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:56.528924 11606 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:09:56.543319 10507 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:56.544651 11660 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:09:56.609351 10430 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.610517 11617 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.623142 10507 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.625783 11646 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.679251 10504 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.680586 11646 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.691547 10432 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.692914 11646 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.788161 10432 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.789589 11647 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.798796 10504 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.800626 11646 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.803701 10430 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.806478 11592 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.853801 10584 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.857021 11684 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:56.934671 10430 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:09:56.948985 10404 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81b70ca03fbe4622a03717efd079d598. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:56.952322 10584 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:56.953794 11642 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:56.955241 10506 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:56.956807 11615 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.959597  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:56.965006 11642 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:56.967231 10584 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:56.968078 10507 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:56.987304 10430 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:56.987389 10333 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.048100 10438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:09:57.049484 10438 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
W20260709 12:09:57.074906 10404 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: deab11eaf55b4f3ebe48926a1e2afa38. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:57.078253 10584 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:57.078636 10506 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:57.081815 11685 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:57.084065 11685 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:57.088199 10587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:09:57.088754 10587 raft_consensus.cc:606] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:57.092816 10504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:09:57.093392 10504 raft_consensus.cc:606] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:09:57.101267 11685 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.102386 10587 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.106200 10584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:09:57.105526 10506 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.106786 10584 raft_consensus.cc:606] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:09:57.111407 11647 raft_consensus.cc:993] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:09:57.111936 11647 raft_consensus.cc:1081] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:09:57.113811 10504 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:09:57.114362 10504 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:57.114732 10504 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.116829 10430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:09:57.117424 10430 raft_consensus.cc:606] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:09:57.122815  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:09:57.124889  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:57.125646  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:57.125466 10506 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:57.129120 10504 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:57.133255 10504 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:57.134569 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:09:57.135408 10430 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.135705 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:09:57.136296 10584 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:57.136590 10584 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:09:57.137408 10584 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:57.138625 10584 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.126340 10336 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.150642 10430 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:09:57.160722 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:57.162082 11684 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:57.163038 11684 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:09:57.164623 11684 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:57.166520 10584 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:09:57.172470 10584 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:57.174046 11643 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:09:57.177583 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b474d9b241d4a698702287ee1bbe3b2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:57.192818 10430 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:57.207194 11592 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:57.207216 11643 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.225466 10579 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.246539 10430 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:09:57.257220 10587 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:57.250164 10336 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:57.288266 10336 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:57.290278 10553 consensus_peers.cc:597] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e4cd5d330f694dacb199bd61266c8f5a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:57.293929 10506 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:57.297353 11647 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:57.298904 10431 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:57.311239 11646 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:57.581629 11704 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:09:58.710872 11714 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.711558 11714 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.713724 11714 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.720368 11717 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.720865 11718 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.721108 11717 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.721550 11713 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.721664 11716 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.722321 11713 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:58.723727 11717 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.724436 11717 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.724709 11713 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:58.725035 11717 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:58.721550 11718 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.727478 11717 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.722352 11716 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:58.727963 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.269s	user 0.025s	sys 0.013s
W20260709 12:09:58.728663 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.270s	user 0.026s	sys 0.013s
I20260709 12:09:58.729823 11716 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.733086 11718 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.737414 11719 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.738786 11719 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.741240 11719 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.738018 11718 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.737785 11716 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.742404 11718 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.742457 11716 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.745648 11718 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.745625 11716 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:58.757058 10504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.433s	user 0.009s	sys 0.039s
W20260709 12:09:58.757750 10504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.434s	user 0.009s	sys 0.039s
I20260709 12:09:58.758556 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1006) took 1436 ms. Trace:
W20260709 12:09:58.764597 10476 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.293s	user 0.023s	sys 0.060s
W20260709 12:09:58.767217 10476 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.295s	user 0.024s	sys 0.060s
I20260709 12:09:58.758939 10504 rpcz_store.cc:276] 0709 12:09:57.321753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:57.321912 (+   159us) service_pool.cc:224] Handling call
0709 12:09:57.322384 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:09:57.322689 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:57.322714 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:09:57.323643 (+   929us) log.cc:844] Serialized 492 byte log entry
0709 12:09:58.758142 (+1434499us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:58.758184 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:58.758269 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:09:58.758325 (+    56us) raft_consensus.cc:1685] finished
0709 12:09:58.758338 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:09:58.758474 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3840,"thread_start_us":1434082,"threads_started":1,"wal-append.queue_time_us":325224,"wal-append.run_cpu_time_us":3270,"wal-append.run_wall_time_us":1016936}
I20260709 12:09:58.771169 11718 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.771821 11718 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.774121 11718 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.777052 11713 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.777768 11713 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:58.782564 11713 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:58.789794 11713 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.789944 11715 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.790383 11713 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.790563 11715 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.789822 11719 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.793524 11719 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.794844 11715 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.791419 11718 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.795650 11719 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.796028 11718 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.797662 11713 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.791646 11717 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.799093 11725 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.799649 11715 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.800405 11715 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.801635 11701 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.803336 11701 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:58.798916 11717 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.799655 11725 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.808723 11713 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.809310 11713 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.811532 11713 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.808074 11719 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.815025 11719 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.818611 11725 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.819362 11720 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:58.820695 11643 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.922851 11723 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:58.854342 11718 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.820067 11720 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:58.923741 11708 raft_consensus.cc:993] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:09:58.925933 11724 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:58.880656 11713 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.928661 11718 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.927884 11643 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:58.925874 11723 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:58.934002 11717 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.947953 11723 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.954223 11708 raft_consensus.cc:1081] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:09:58.956852 11727 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.934433 11718 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:58.932292 11713 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:58.958295 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.505s	user 0.048s	sys 0.061s
I20260709 12:09:58.958572 11643 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:58.959628 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.480s	user 0.003s	sys 0.008s
W20260709 12:09:58.968484 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.515s	user 0.049s	sys 0.061s
W20260709 12:09:58.968919 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.515s	user 0.005s	sys 0.008s
I20260709 12:09:58.939043 11724 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:58.947969 11720 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:58.955313 11723 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:58.969776 11720 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:58.960088 11717 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.016906 11723 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.019328 11713 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.020157 11720 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.022562 11720 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.027905 11732 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.028713 11732 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.041433 11732 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:58.967713 11727 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.046471 11727 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.025892 11643 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.050707 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 976) took 1735 ms. Trace:
I20260709 12:09:59.051192 10430 rpcz_store.cc:276] 0709 12:09:57.315452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:57.315563 (+   111us) service_pool.cc:224] Handling call
0709 12:09:57.316056 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:09:57.316938 (+   882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:57.316968 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:09:57.317978 (+  1010us) log.cc:844] Serialized 492 byte log entry
0709 12:09:57.429713 (+111735us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:57.429774 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:57.429896 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:09:59.050427 (+1620531us) raft_consensus.cc:1685] finished
0709 12:09:59.050449 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:09:59.050604 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":111187,"threads_started":1,"wal-append.queue_time_us":142207,"wal-append.run_cpu_time_us":3979,"wal-append.run_wall_time_us":1124292}
I20260709 12:09:59.053321 11708 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.053958 11708 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.068702 11724 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.070513 11723 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.026302 11717 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.053987 11718 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:09:59.083707 10401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.628s	user 0.008s	sys 0.008s
I20260709 12:09:59.067219 11643 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.087272 11643 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.066181 11708 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:09:59.097599 10401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.644s	user 0.009s	sys 0.008s
I20260709 12:09:59.100355 11726 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.100888 11713 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.099360 11642 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.111441 11720 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.111673 11734 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.112298 11720 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.101080 11726 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.119077 11726 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.120568 11726 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.121332 11726 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:59.124369 11726 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.104861 11718 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.105763 11643 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.107323 11708 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.109799 11717 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.111734 11740 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:09:59.101598 11713 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:59.112409 11734 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.157232 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.157579 11734 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.132113 11724 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.142802 11737 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.157752 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.125135 11642 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.144055 11643 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.161679 11735 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.162365 11731 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.163430 11724 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.173094 11721 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.176618 11728 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.177331 11728 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.141901 11718 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.185707 11724 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.186321 11738 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:59.187526 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.189152 11701 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.181968 11726 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.145581 11708 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.166246 11740 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.197170 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.174479 11720 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.198065 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.199774 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.202150 11741 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.202235 11739 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.194039 10510 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:59.212218 11728 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.212980 11728 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.213598 11728 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.180383 11737 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.231138 11733 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.232273 11743 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.232017 11733 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.233071 11743 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:09:59.235420 11733 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.236269 11743 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.237541 11733 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.237850 11743 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.238706 11744 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.238257 11733 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.226926 11746 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.239346 11744 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.239583 11746 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.238526 11743 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.173966 11721 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.210052 11738 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.206568 11726 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.265996 11748 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.269459 11745 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.241250 11733 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.265547 11751 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.227589 11701 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.215178 11741 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.192528 11735 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.275362 11713 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.339315 11720 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.193107 11731 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:59.215201 11739 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.354458 11642 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.353942 11744 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.350420 11701 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:59.372010 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 143839us
I20260709 12:09:59.373719 11747 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.374634 11731 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.374579 11747 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.375563 11731 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.376192 11731 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.377725 11747 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.345252 11745 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.383443 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:09:59.385113 11753 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.385582 11744 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:09:59.386006 11744 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.385725 11753 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.386525 11744 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:59.395383 11740 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.357550 11720 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.359978 11742 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.384765 11643 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.397702 11728 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.396402 11749 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.378724 11750 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.423134 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.423700 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.426827 11726 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.427443 11726 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.428042 11735 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.406793 11708 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.431780 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.427354 11757 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.345017 11748 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.353554 11751 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.428251 11726 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.415544 11742 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.607592 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.431978 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.610206 10512 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:09:59.600584 11739 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.530634 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.608414 11741 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.612674 10511 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:09:59.621585 11737 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.525852 11746 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.540014 11748 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.544749 11731 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.421620 11701 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.420874 11728 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.437731 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:09:59.435626 11757 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.526082 11743 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.422407 11718 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:09:59.701020 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1060 after lost signal to thread 9264
I20260709 12:09:59.433465 11749 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.650544 11720 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.669171 11728 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.432533 11738 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.691054 11742 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.423206 11754 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.711745 11753 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.425091 11758 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.654004 11744 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.516727 11750 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.689682 11757 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:09:59.731271 11746 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.748701 11745 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.732611 11741 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.748035 11726 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.746886 11708 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.748808 11719 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.755960 11755 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.759702 11741 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.755707 11746 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.782721 11752 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.783651 11752 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.755146 11738 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:09:59.756510 11720 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:09:59.751211 11753 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.772603 11750 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.765187 11721 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:09:59.756999 11758 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:09:59.817453 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1740 after lost signal to thread 10370
I20260709 12:09:59.753727 11754 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:09:59.819093 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1800 after lost signal to thread 10384
I20260709 12:09:59.779677 11708 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:09:59.803820 11738 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.804230 11720 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:09:59.809886 11753 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.012600 11749 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.013376 11749 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.013831 11738 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.013949 11749 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.986374 11731 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:09:59.990970 11744 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:09:59.779910 11755 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:00.024807 11752 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.029927 11755 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:00.023412 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5b60 after lost signal to thread 10462
I20260709 12:10:00.035209 11742 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:00.036296 10553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.037s	sys 0.089s
W20260709 12:10:00.037534 10553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.038s	sys 0.089s
W20260709 12:10:00.038760 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.024s	sys 0.099s
W20260709 12:10:00.039358 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.025s	sys 0.099s
I20260709 12:10:00.022929 11746 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:00.051076 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.044s	sys 0.083s
W20260709 12:10:00.051605 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.045s	sys 0.084s
W20260709 12:10:00.052098 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 557872us
I20260709 12:10:00.034242 11728 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.034301 11752 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.022653 11731 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.041030 11748 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:00.112712 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.023s	sys 0.023s
I20260709 12:10:00.039815 11742 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:00.113163 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.023s	sys 0.023s
I20260709 12:10:00.114289 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.119288 10587 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.098481 11741 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.070355 11746 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.110390 11749 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:00.127235 10551 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.069s	sys 0.112s
W20260709 12:10:00.127858 10551 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.069s	sys 0.112s
I20260709 12:10:00.146351 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:00.151382 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4120 after lost signal to thread 10537
W20260709 12:10:00.151831 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08001c8ca0 after lost signal to thread 10538
I20260709 12:10:00.152315 11742 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:00.153134 11245 debug-util.cc:398] Leaking SignalData structure 0x7b0800312960 after lost signal to thread 10611
W20260709 12:10:00.153494 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6e80 after lost signal to thread 10612
W20260709 12:10:00.154767 11245 debug-util.cc:398] Leaking SignalData structure 0x7b0800348ae0 after lost signal to thread 11248
I20260709 12:10:00.157812 11758 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.160651 11764 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.162150 11735 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.162626 11749 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.163684 11728 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.164367 11742 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.164987 11731 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.125653 11762 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.127995 11741 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.112068 11748 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.099905 11752 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.160713 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:00.181043 11245 debug-util.cc:398] Leaking SignalData structure 0x7b080011f120 after lost signal to thread 11249
I20260709 12:10:00.198541 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.209844 10583 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.128655 11746 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.207615 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:00.166055 11715 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.221349 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.222061 10508 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.222430 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.223084 10512 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:00.208732 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.142027 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.210816 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.212785 10430 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:00.211688 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83212.6us
I20260709 12:10:00.219719 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.197214 11708 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.229543 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.231245 10507 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.228480 11762 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.234571 11762 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.236179 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.236698 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.210856 11753 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.237545 10583 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.238636 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.220300 11720 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.245047 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.247468 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.247413 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.248690 10583 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.248673 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.249662 10430 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.200913 11749 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.223868 11769 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.210659 11742 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.209069 11728 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.257522 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.240111 11762 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.268126 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.269143 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.269366 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.269891 10436 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.270161 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.272642 11746 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.273451 11746 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.274420 11772 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.244885 11753 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.239554 11754 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.240118 11736 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.220793 11741 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.278163 11770 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.280670 11741 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.208230 11735 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:00.281997 11742 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.282081 11748 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.251999 11768 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:00.267952 11762 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.261206 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.219425 11764 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.274147 11746 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.262373 11731 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:00.265072 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.262547 11769 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.298085 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.298792 11219 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.299299 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.301189 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.301738 11765 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.275215 11772 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.289425 11753 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.304800 11764 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.305814 11772 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.305583 11731 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.306351 11753 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.307492 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.310374 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.307754 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.299911 11218 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.302448 11765 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.280692 11751 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.280710 11752 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:00.312927 10438 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.333738 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.292838 11736 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.334246 11735 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.295791 11771 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.333622 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.335279 11771 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.333927 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.336226 10513 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.340505 11220 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57892
I20260709 12:10:00.341506 11220 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:00.342036 11220 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:00.343422 10479 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.506s (SENT) [suppressed 2 similar messages]
I20260709 12:10:00.307060 11742 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.346083 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 224) took 1667 ms. Trace:
I20260709 12:10:00.347493 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.309921 11741 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.347993 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.348626 11741 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:09:59.832834  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:00.352139 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.296113 11754 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:00.345913 11742 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.355123 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.366539 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.366436 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.278825 11770 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.362037 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.410310 10333 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 229) took 1702 ms. Trace:
I20260709 12:10:00.365340 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.411468 10436 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:00.412343 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 215) took 1376 ms. Trace:
I20260709 12:10:00.346616 10336 rpcz_store.cc:276] 0709 12:09:58.678884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.959154 (+280270us) service_pool.cc:224] Handling call
0709 12:10:00.345926 (+1386772us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:10:00.412719 10335 rpcz_store.cc:276] 0709 12:09:59.036076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.036329 (+   253us) service_pool.cc:224] Handling call
0709 12:10:00.412270 (+1375941us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
I20260709 12:10:00.411012 10333 rpcz_store.cc:276] 0709 12:09:58.656841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.773439 (+1116598us) service_pool.cc:224] Handling call
0709 12:10:00.359246 (+585807us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":571070,"reported_tablets":1}
I20260709 12:10:00.419591 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.419881 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.420289 10583 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.293335 11768 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.243817 11708 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:00.419014 10579 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:00.425493 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.364511 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.377652 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.427897 11749 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.428289 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.429545 10438 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:00.429955 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.324213 11753 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.363654 11736 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.432636 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.437176 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.374140 11746 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.379348 11769 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.411569 11731 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.456907 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:00.447036 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 103729us
I20260709 12:10:00.449415 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.457984 10438 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.452747 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.445142 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.459831 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.461732 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.461830 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.449041 11217 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.458469 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:00.471218 11220 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.472101 11747 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:00.472575 11747 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.472966 11747 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:00.452297 10511 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.419927 11754 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:00.453085 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.463853 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.474336 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.505218 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.633600 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.505020 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.539857 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.622524 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.507025 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.535784 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.682264 10578 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.628085 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.642484 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:00.630456 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.706182 10512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:00.535602 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.680625 10583 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:00.711838 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 69501.9us
I20260709 12:10:00.507588 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.708549 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.510270 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.717087 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.629263 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.744300 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.795965 10583 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:00.538301 11765 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.538962 11728 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.498067 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.476861 11214 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:00.668406 11723 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:00.714027 11736 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:00.790798 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71695.3us
I20260709 12:10:00.792997 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.793998 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.504784 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.538908 11742 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.629782 11768 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.800818 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:00.800292 10580 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:00.503974 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.679513 11741 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:00.796051 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.456416 11753 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:00.750572 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.787900 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.872128 10585 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:00.861755 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.808713 11771 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:00.875779 11755 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:00.797271 11216 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:00.899293 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.056360 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.878630 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.924405 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.929557 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.069623 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.898494 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.071148 11221 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:01.073112 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.074190 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:00.997593 11780 ts_tablet_manager.cc:933] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.082585 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.082942 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.898886 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.873260 11685 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.103093 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.103828 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.104166 11733 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.104643 11733 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.105038 11733 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.105091 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.105837 10578 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:01.106573 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:00.924028 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.929405 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:00.925549 11754 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.078918 10511 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:01.100251 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.109174 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:00.889248 11220 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.069481 11736 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.077523 11217 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:01.115429 11720 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.120064 11212 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:01.120604 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.120707 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.121157 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.121790 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.123514 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.070573 11723 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.107762 11770 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.125538 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.126425 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.121002 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.129942 10510 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:01.119088 11736 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.131108 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.099246 11755 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.131937 10430 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:01.133333 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.117096 11685 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.136848 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.130568 10429 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:01.124050 11720 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.155751 11720 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.154860 11736 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.157581 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.157871 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.158870 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.159724 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.160099 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.160466 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.161298 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.162448 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.162562 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.163437 11762 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.142051 10438 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:01.164577 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.165740 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.166213 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.166826 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.157912 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.141656 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.140888 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.143276 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.170469 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:01.143721 11755 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.144373 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.145267 10512 raft_consensus.cc:606] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:01.145498 11753 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.147212 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.150523 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.175098 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.151794 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:01.152593 11708 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.001s
W20260709 12:10:01.154096 11747 consensus_meta.cc:287] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.681s	user 0.001s	sys 0.010s
I20260709 12:10:01.163358 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.142530 11685 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.168553 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.167912 10436 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:01.171670 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.182355 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.182839 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.183383 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.188910 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.185087 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.191838 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.192495 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.192605 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.194070 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.195279 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.192235 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.196790 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.194944 11742 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.175671 10434 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:01.197973 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.197639 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.199849 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.189018 10430 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:01.201962 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.201643 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.201996 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.198280 11742 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.211972 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.212952 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.214546 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.204989 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.215570 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.215939 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.202844 11754 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.218297 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.217361 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.217805 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.209801 11765 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.188621 11747 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.211169 11742 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.228585 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.199098 11768 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:10:01.229454 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.230393 10429 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:01.226095 11736 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:01.232429 11768 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:01.232895 11768 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:01.234083 11768 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.228586 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:01.231091 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:01.242972 11708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36736 (request call id 6) took 3982 ms. Trace:
I20260709 12:10:01.243758 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.235810 11723 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:01.245864 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.245308 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.218346 11754 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.228677 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.260903 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.223717 11765 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.263922 11685 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.257372 11780 tablet_copy_client.cc:323] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:10:01.278954 11779 ts_tablet_manager.cc:933] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:01.280473 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.292064 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.269795 11747 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:01.271190 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:01.275137 11768 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.299834 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.308696 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.309971 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.247014 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:01.311143 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.311690 11753 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:01.311599 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 84993.7us
I20260709 12:10:01.312346 11753 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:01.312788 11753 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.275928 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.277053 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.270511 11733 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.266669 11736 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.262616 11754 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.260598 10505 raft_consensus.cc:606] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:01.286716 11765 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.258638 11785 ts_tablet_manager.cc:933] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:01.296048 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.295863 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.286309 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.297446 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.325755 10430 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
I20260709 12:10:01.299081 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.297784 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.306404 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.299486 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:01.264066 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52014.6us
I20260709 12:10:01.301537 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.328986 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.295595 11685 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.329756 11685 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.334267 11765 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.316905 11736 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.335943 11736 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.336217 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.339082 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.339253 10430 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:10:01.315505 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:01.282279 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.288327 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.274137 11720 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.292934 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:01.316614 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 125324us
I20260709 12:10:01.279536 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.291074 11783 ts_tablet_manager.cc:933] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.299875 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.288691 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.321412 10583 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:01.294431 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.303411 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.299401 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:01.281463 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.347258 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.348658 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.347937 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.351430 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.365952 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.371691 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.352643 11747 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.348733 11787 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.365924 11760 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.381532 11760 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.381353 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.382073 11760 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.378885 11747 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.398576 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.379065 11787 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.405208 11747 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.409862 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.399637 10434 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:01.426810 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.410135 11787 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.411373 11785 tablet_copy_client.cc:323] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:10:01.431579 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.432977 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.433560 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.438596 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.439181 10578 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:01.439476 10578 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:01.404552 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:01.437170 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.445365 10438 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.448788 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.452967 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.453395 11720 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:01.454391 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.449026 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.449568 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.449743 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.410920 11788 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.468115 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.469182 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.425074 11755 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.247012 11708 rpcz_store.cc:276] 0709 12:09:57.254759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:57.254869 (+   110us) service_pool.cc:224] Handling call
0709 12:09:57.261299 (+  6430us) log.cc:844] Serialized 492 byte log entry
0709 12:09:57.317334 (+ 56035us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b54003d4120. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a54dbe 00007f2c92a5f930 00007f2c92a5f8d8 00007f2c92a5f858 00007f2c92a5f813 00007f2c92a5ea9c 00007f2c9894a2f6 00007f2c98944aa8 00007f2c92a9b7cb 00007f2c92a9e2c0
0709 12:10:01.237542 (+3920208us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":231,"raft.queue_time_us":296,"raft.run_cpu_time_us":1727,"raft.run_wall_time_us":1726,"spinlock_wait_cycles":2984704,"thread_start_us":25495,"threads_started":1,"wal-append.queue_time_us":22293,"wal-append.run_cpu_time_us":4419,"wal-append.run_wall_time_us":1034152}
I20260709 12:10:01.467939 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:01.460004 10578 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.490477 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:01.480434 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.483124 10429 raft_consensus.cc:606] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:01.483210 11755 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:01.508618 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 156486us
I20260709 12:10:01.491789 11733 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:01.520884 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 96248.1us
W20260709 12:10:01.522581 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 123945us
I20260709 12:10:01.500319 11765 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.526693 11783 tablet_copy_client.cc:323] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:10:01.510468 11755 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.529146 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.531726 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet e4cd5d330f694dacb199bd61266c8f5a from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:10:01.532485 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet e4cd5d330f694dacb199bd61266c8f5a from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-e4cd5d330f694dacb199bd61266c8f5a
I20260709 12:10:01.535104 10552 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.535475 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.541711 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.543022 11770 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.543164 11762 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.544034 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.544914 11762 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.546953 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.547868 11762 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.549382 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.552057 11762 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.529275 11779 tablet_copy_client.cc:323] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:10:01.564242 11788 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.564921 11788 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.550288 10583 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:01.568859 10552 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.538568 11765 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.579742 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.581657 10552 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.582808 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.578888 10335 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 1 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:01.571144 11796 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.554802 11742 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.593523 11708 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.598172 10597 tablet_copy_source_session.cc:215] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:10:01.586894 11790 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.599021 11685 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.599974 11749 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.578603 11768 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.614697 11787 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:01.632892 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 121554us
I20260709 12:10:01.617385 11753 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.592377 11796 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:01.641767 11736 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.643347 11747 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.644815 11753 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.645347 11787 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.646327 11215 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:01.634894 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.634617 11780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cd5d330f694dacb199bd61266c8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:01.654354 11799 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:01.647739 11796 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.657042 10578 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:01.651563 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 237050us
I20260709 12:10:01.649145 11742 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:01.659916 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 130789us
I20260709 12:10:01.651779 11786 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.665546 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:01.675207 10578 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:01.671278 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.676932 11177 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:01.677896 11747 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.678067 11753 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:01.678557 11768 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.678648 11753 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:01.680040 11753 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.681619 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:01.664054 11786 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.691200 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.670270 11742 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.696949 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:01.670616 11754 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:01.696405 11768 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.697844 11728 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.698372 11768 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.698843 11768 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.704089 11736 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.704995 11754 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:01.707298 10582 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.709344 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:01.702792 11760 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.697232 11786 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.692349 11788 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:01.697058 11742 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:01.721824 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49563.3us
I20260709 12:10:01.715016 11801 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.720183 11736 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:01.728080 11801 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.727393 11742 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.729360 11801 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.728817 11685 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.722726 11754 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:01.731789 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 149070us
I20260709 12:10:01.731992 10578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:10:01.734359 10578 raft_consensus.cc:606] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:01.735999 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.769840 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.772275 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.779343 11765 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.779841 11755 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.782200 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:01.782938 10429 raft_consensus.cc:606] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.734848 11754 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:01.794291 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.795145 11754 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.795687 11804 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.796543  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:01.796808 11805 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:01.803855  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:01.776693 11787 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:01.800542 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 137320us
I20260709 12:10:01.803814 11804 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:01.804875  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:01.806515 11788 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.807719 11755 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.803844 11805 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:01.813370 11804 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:01.839265 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45085.3us
I20260709 12:10:01.839929 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.848227 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.848521 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.849426 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.851363 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.840991 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.840943 11806 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.855712 11806 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:01.856107 11806 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.838469 11728 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.856495 11805 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.858655 11749 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.861835 11791 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.865860 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.867448 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.868175 11791 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:01.868633 11786 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:01.869560 11801 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.872094 10552 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.872401 11728 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.873257 11791 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.858163 11787 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.876550 11786 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:01.873457 11728 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:01.877674 11728 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.879571 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.880828 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.878517 11805 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.885790 11178 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.890317 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.892854 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.893476 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.895251 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.896335 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.898470 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.898562 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.900053 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.902004 11801 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:01.902263 11749 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.903261 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.903599 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.903141 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.904505 10505 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:01.904670 11753 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.905627 11178 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.905939 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.907250 11753 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.907490 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.909921 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.910468 10512 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.907636 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:01.912047 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.912894 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:01.914204 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:01.917842 11178 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:01.919194 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 146992us
I20260709 12:10:01.924857 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.926218 10505 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:01.928359 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.928682 11178 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.930318 11178 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:01.922118 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:01.926139 11762 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.883567 11787 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:01.932525 11770 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.948167 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.949005 10504 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.950886 11762 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.951179 11799 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:01.945425 10333 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:01.933614 11753 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.877055 11768 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.876924 11754 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.967555 11803 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:01.973881 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.972999 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.884159 11804 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:01.974915 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.975577 10512 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:01.978631 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.988691 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.979202 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.980298 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.989876 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.990154 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.980571 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.974126 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:01.966321 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:01.983610 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.984047 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.984880 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.985030 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.958676 11760 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:02.002285 10437 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.985809 11799 raft_consensus.cc:993] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:10:02.004954 11754 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:01.977744 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.006349 10429 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:01.985883 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:01.991218 10513 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.968719 11753 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:01.990592 10430 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:01.992064 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:01.993117 10504 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:02.010555 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:01.982880 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.018950 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.020462 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.011258 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.021533 10583 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.022644 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.012677 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.015311 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.015900 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.015947 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.024624 10506 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.010911 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.025038 10584 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:02.026326 10586 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.038249 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.039259 10579 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:02.024049 10585 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.044483 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.045337 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.044576 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.009908 11801 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.051998 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.018465 11753 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.017890 11770 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.002811 11806 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.042346 10511 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.054060 10584 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.061751 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.065435 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.042030 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.076522 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.077519 10581 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.079967 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.060475 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.082015 10583 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.008697 11796 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.065519 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.071985 11803 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.029152 11810 raft_consensus.cc:2764] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 12:10:02.047093 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 80934.9us
I20260709 12:10:02.092561 11804 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:02.093642 11804 raft_consensus.cc:1081] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:10:02.087312 11803 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:02.096485 11803 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.097359 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.098587 11803 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.063009 11801 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.101110 11770 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.108016 11768 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:02.112645 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.122287 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.122030 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.123634 10584 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
W20260709 12:10:02.125066 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 138352us
W20260709 12:10:02.126727 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 228124us
I20260709 12:10:02.127192 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.128687 10584 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.136518 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.142176 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.151479 11806 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.088747 11742 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.153224 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.163091 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.163129 10584 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:02.168659 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.169375 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.170117 10431 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:02.171868 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:02.173549 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 243306us
I20260709 12:10:02.174633 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.176345 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.176738 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:02.174257 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.177160 10438 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.178612 10436 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.178963 10437 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:02.180292 10585 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:02.182205 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.182485 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:02.182955 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.183123 10429 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.183656 10432 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.186331 10430 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.186367 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.189031 10430 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.190819 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.194073 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.195583 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.170682 10511 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:02.199261 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.201542 10506 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.221777 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.180178 11770 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.227442 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.220147 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.203823 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:02.240929 11768 raft_consensus.cc:993] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:02.241667 11768 raft_consensus.cc:1081] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:02.242806 11742 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:02.247830 10552 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.246230 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:02.251399 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.229473 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:02.256964 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.265662 11787 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.266173 11787 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.266227 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.266614 11787 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.254545 11212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57892
I20260709 12:10:02.267802 11212 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:02.268318 11212 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.269125 11790 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.270706 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.255187 11806 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.254905 11708 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.274542 10585 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:02.276405 10552 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.272841 11790 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.280575 11806 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.285092 11762 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:02.285916 11753 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.272683 11820 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.292587 11820 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.293309 11820 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.279443 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.306901 11749 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.225835 11728 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:02.319235 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93256.9us
I20260709 12:10:02.252169 11768 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.320123 11736 raft_consensus.cc:993] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:02.337466 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.343243 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:02.343402 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 121651us
I20260709 12:10:02.249014 11819 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:10:02.344393 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:02.354954 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:02.371610 11801 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.369560 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:02.374709 10432 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:02.374969 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:02.432791 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 88486.2us
I20260709 12:10:02.450562 11820 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.470217 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.360384 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:02.356093 11819 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.471308 11819 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:02.471819 11820 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:02.472549 11805 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.009s
I20260709 12:10:02.346431 11754 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.112s
I20260709 12:10:02.472617 11819 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:02.474493 11819 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.351354 11802 raft_consensus.cc:1081] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
W20260709 12:10:02.449781 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 95385.2us
I20260709 12:10:02.346855 11728 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.456621 11796 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.344684 11742 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.475651 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.439155 11786 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.476477 11786 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:02.476966 11786 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.477799 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 117528us
I20260709 12:10:02.351616 11685 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.458511 11770 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.464339 11753 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:02.355988 11749 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:10:02.375597 11220 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.467077 11787 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:02.469177 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:02.475360 11780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57892 (request call id 191) took 2120 ms. Trace:
I20260709 12:10:02.478593 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.489085 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.491678 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.491441 11802 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.492971 10505 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:02.498976 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.500072 11802 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.490056 11742 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.501101 11802 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.502624 11790 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.502825 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.503189 11790 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:02.486943 11780 rpcz_store.cc:276] 0709 12:10:00.354689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.354821 (+   132us) service_pool.cc:224] Handling call
0709 12:10:01.086445 (+731624us) ts_tablet_manager.cc:934] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
0709 12:10:01.617641 (+531196us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:02.054344 (+436703us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:02.063408 (+  9064us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:02.474031 (+410623us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:02.474589 (+   558us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:02.475135 (+   546us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:02.475239 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":433,"tablet-copy.queue_time_us":619377,"thread_start_us":196872,"threads_started":2}
I20260709 12:10:02.507155 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.508116 11685 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:02.508749 11214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57890
I20260709 12:10:02.509656 11214 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:02.510253 11214 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.511329 11787 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.516253 11780 tablet_copy_client.cc:806] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:10:02.517391 11780 tablet_copy_client.cc:670] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
W20260709 12:10:02.519533 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 49379us
I20260709 12:10:02.520692 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.522189 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.482018 11805 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.524536 10505 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:02.525429 10522 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 0b474d9b241d4a698702287ee1bbe3b2 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:10:02.524709 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.526356 10522 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 0b474d9b241d4a698702287ee1bbe3b2 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-0b474d9b241d4a698702287ee1bbe3b2
I20260709 12:10:02.528164 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.533108 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:02.534493 11805 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.541404 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:02.542943 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56335.6us
I20260709 12:10:02.543766 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.543192 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:02.544448 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:02.547297 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.547950 10432 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
I20260709 12:10:02.548508 10586 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:10:02.548906 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.549098 10586 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:02.549537 10586 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.549854 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.550275 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.550711 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.541651 11813 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.552323 11813 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.552901 11813 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.554491 11773 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.555609 10584 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.557685 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.558276 10430 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.561726 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:02.562120 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.562924 10432 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:02.558209 11728 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:10:02.562605 10511 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:02.566856 10430 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:02.568889 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.570801 11768 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:02.571310 11780 tablet_copy_client.cc:538] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:10:02.504910 11790 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:02.567327 11820 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.584316 11820 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.584949 11820 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.585544 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 78473.6us
I20260709 12:10:02.573554 11742 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.540776 11815 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.574163 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:02.599362 11815 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.600260 11768 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:02.600893 11768 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:02.600275 11815 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.582321 10586 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.557289 11803 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:02.599960 11824 raft_consensus.cc:993] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:02.612077 11803 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.612740 11803 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.613188 11770 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:02.602222 11768 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.613828 11754 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.614426 11754 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:02.614065 11770 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.587795 11796 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.619495 11796 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:02.620189 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46107.4us
I20260709 12:10:02.621309 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:02.620873 11796 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.622316 11796 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.623296 11810 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.609037 10586 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.587715 10429 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:02.633654 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.636612 10436 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.637492 10430 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.622686 11768 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.640516 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:02.642756 11221 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:02.644169 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:02.646661 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 105330us
I20260709 12:10:02.657704 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.651115 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.656391 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:02.624409 11810 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.659456 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:02.660378 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.658174 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.645726 10448 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet deab11eaf55b4f3ebe48926a1e2afa38 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:10:02.666329 10478 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.673552 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:02.673298 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.674758 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:02.675321 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.645793 11825 raft_consensus.cc:1081] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:02.678139 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:02.671161 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.670316 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.679895 10432 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.683449 10587 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:02.681001 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.681792 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.649057 11824 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:02.682451 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.687711 10580 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:02.688166 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:02.689741 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.653795 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:02.685910 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.649765 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.667994 11799 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.686965 10438 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:02.690693 10581 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:02.699221 10448 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet deab11eaf55b4f3ebe48926a1e2afa38 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-deab11eaf55b4f3ebe48926a1e2afa38
I20260709 12:10:02.707283 11822 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.708853 11790 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.688344 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:02.713629 11827 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.696704 11753 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:10:02.716426 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.717037 11753 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:02.717646 11753 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:02.719373 10447 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet 81b70ca03fbe4622a03717efd079d598 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:10:02.721693 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.722033 10513 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.722502 10581 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:02.722558 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.685773 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.692874 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:02.647433 11728 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:10:02.685773 10586 raft_consensus.cc:2368] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:02.697059 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.698257 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:02.720198 10447 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet 81b70ca03fbe4622a03717efd079d598 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-81b70ca03fbe4622a03717efd079d598
I20260709 12:10:02.735368 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.723418 10583 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:02.699717 10578 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:02.701805 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.671649 11804 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.689843 11814 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.753798 11825 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.743146 11804 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.755718 11804 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.684965 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:02.698482 11818 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:02.758600 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47799.6us
I20260709 12:10:02.766583 11818 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.767264 11818 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.737289 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51598.7us
I20260709 12:10:02.770859 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.724406 10430 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:02.739406 10584 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.739318 11685 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:02.741793 10522 tablet_copy_source_session.cc:215] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:10:02.701179 11799 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.737665 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.778443 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.778680 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.677120 11212 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.746482 11821 raft_consensus.cc:2764] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:02.747396 11728 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.685186 11791 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:10:02.752170 11762 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.710721 11742 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.729755 11794 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:02.759413 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 62516.6us
W20260709 12:10:02.760125 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 75251.9us
I20260709 12:10:02.801590 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.730206 10429 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:02.745049 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72541.4us
I20260709 12:10:02.749444 10448 tablet_copy_source_session.cc:215] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:10:02.779520 11799 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.769186 11770 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:02.802795 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.807377 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:02.775031 11806 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.779747 11755 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.729612 11813 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.777927 11825 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.800140 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:02.792778 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.812673 11794 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.694641 11754 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.812383 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.817826 10510 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.797456 11742 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.782310 11802 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.821187 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.781786 11728 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:02.823323 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 79068.3us
I20260709 12:10:02.784560 11827 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.830011 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.781744 11824 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.832077 10510 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:02.825170 11822 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.838809 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.828136 10505 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.825736 11825 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.841444 10510 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.850505 11822 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:02.852819 10511 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.815301 11786 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.784765 11753 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.835669 11810 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:02.865767 11806 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.884199 11212 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:02.836504 11742 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.867048 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:02.870383 10510 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.839589 11814 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:10:02.871404 11770 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.871503 11822 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.872383 11827 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.870028 11754 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.879243 11786 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:02.891917 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 70837.3us
I20260709 12:10:02.903146 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.904572 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.855369 11794 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.871079 11825 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.899526 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 84508.1us
W20260709 12:10:02.841951 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 69839.5us
W20260709 12:10:02.912084 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:02.914578 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:02.915474 10580 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:02.915863 10580 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:02.917166 10580 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.919405 10580 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.936208 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 123950us
I20260709 12:10:02.936376 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.932992 11827 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.951995 10585 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:02.935617 11794 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.946614 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:02.990175 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:02.948040 11813 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:02.949452 11780 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:10:02.937390 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 107465us
I20260709 12:10:02.992944 11736 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.995653 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.996225 11773 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:02.996975 11773 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:02.998050 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.902611 11214 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.982703 11807 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.954293 11803 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.994000 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:02.994014 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:02.999321  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:02.903692 11728 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:02.954361 11820 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:02.988474 11787 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:03.009075 11773 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.011281 11749 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.025591 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:03.005026 11810 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.007179 11827 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.059556 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:03.019220 11787 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:03.052028 11806 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.083938 11753 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.000746 11815 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.020877 11791 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:10:03.069029 11755 raft_consensus.cc:993] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:10:03.049896 11828 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:10:03.023725 11814 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:10:03.067549 11824 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.105336 11824 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:03.106755 11824 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.111101 11824 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.123605 11787 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.125583 11787 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.128111 10504 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.092768 11803 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.141247 11214 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.067648 10510 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:03.149701 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 120565us
W20260709 12:10:03.149871 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 155965us
I20260709 12:10:03.150686 10580 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:03.132205 11804 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.150892 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:03.152196 11802 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.152902 11791 raft_consensus.cc:3055] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.161754 11791 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:03.162564 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.163388 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:03.164649 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:03.154192 11828 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.157472 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.159403 11814 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.160363 11755 raft_consensus.cc:1081] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:10:03.155578 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:03.176779 11807 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.163182 11791 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.191741 11754 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:03.192966 11214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57904
I20260709 12:10:03.193333 11791 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.193643 11753 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.193984 11214 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:03.199366 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.203186 10580 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.176224 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 271718us
I20260709 12:10:03.208705 10478 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:03.165906 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106501us
I20260709 12:10:03.186964 11736 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:03.178339 11814 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:03.175544 11828 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:03.216563 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:03.226087 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:03.177871 11796 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:03.231389 10509 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.181915 11802 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:03.239560 11770 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.200717 11753 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:03.165024 11803 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:10:03.210072 11214 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.243095 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.247593 11783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b474d9b241d4a698702287ee1bbe3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:03.247227 11823 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:10:03.243860 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:03.254973 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.219437 11754 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:03.257575 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.257954 11805 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:10:03.267155 11834 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.249572 11753 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:03.269912 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.271978 11803 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.272925 11814 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.280496 11835 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.262884 11760 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:10:03.278827 10580 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:03.325104 11755 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.325716 11804 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.307099 11754 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.269958 11794 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:10:03.304855 11768 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:03.314920 10429 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:03.286861 11822 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.309958 11803 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:03.317485 11814 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.240082 11828 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.287845 11802 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:03.301723 11728 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.293409 11820 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.356467 11768 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.325500 11177 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:03.325721 11755 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:03.395924 11755 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.333039 11835 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:03.263556 11818 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:03.354489 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48995.9us
I20260709 12:10:03.371573 11805 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.376014 11803 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.381157 11825 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.400717 11835 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:03.404039 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 78726us
I20260709 12:10:03.405161 11770 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.412889 11828 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.386116 11815 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.414821 11755 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.392465 11768 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:03.403759 11742 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.010s
I20260709 12:10:03.435910 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:03.436200 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:03.436529 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:03.437484 10511 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.438297 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.418255 11805 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:03.433759 11770 raft_consensus.cc:697] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:03.449477 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:03.451303 11770 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.444948 11742 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.447683 11786 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.464466 11792 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.446134 10447 tablet_copy_source_session.cc:215] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:10:03.449292 11803 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.501607 10510 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:03.507822 10511 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
W20260709 12:10:03.498356 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47916.3us
I20260709 12:10:03.503237 11820 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:03.547410 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:03.543911 10584 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.028s	sys 0.072s
I20260709 12:10:03.522396 11792 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:03.557902 10584 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.028s	sys 0.072s
W20260709 12:10:03.555943 11799 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.009s
W20260709 12:10:03.573280 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.558857 11820 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.584528 11742 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:03.561064 11805 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.589365 11805 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.610517 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:03.610642 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 397958us
I20260709 12:10:03.614013 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.615501 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.617857 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.622018 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:03.626950 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 280435us
I20260709 12:10:03.632441 11728 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:03.633015 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:03.633787 10511 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:03.633152 11728 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:03.633222 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:03.633494 11838 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:03.637322 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:03.638180 11728 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:03.603068 11825 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.639786 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:03.646592 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.648417 10584 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.654422 11804 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.655210 11804 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:03.656368 11804 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.658035 11804 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.630649 11736 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:03.671397 11829 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.651275 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.672152 11818 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.601480 11799 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.646932 11792 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:03.680953 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.682622 11796 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:03.683445 11796 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.677376 11736 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.692994 11822 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.646149 11815 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:03.703383 11728 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:03.703657 11796 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:03.704022 11831 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:03.704105 11728 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.725538 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:03.672165 11829 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:03.728545 10510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9042bec741aa476587972e688997da43"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
W20260709 12:10:03.717444 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.731614 10510 raft_consensus.cc:606] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:03.729390 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:03.734269 11799 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.738873 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.741250 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.739733 10584 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.740206 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.705015 11831 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.744210 11830 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.744787 11830 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:03.739923 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.750424 10579 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.745404 11830 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.759253 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.759986 10584 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.760280 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.760656 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.761337 10585 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:03.762296 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:03.737581 11829 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.745947 11790 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.032s	user 0.010s	sys 0.010s
W20260709 12:10:03.773222 11214 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.505s	user 0.015s	sys 0.000s
I20260709 12:10:03.779666 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:03.773371 11791 consensus_meta.cc:287] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.011s
W20260709 12:10:03.798542 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.779672 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:03.807611 11824 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.694s	user 0.003s	sys 0.017s
I20260709 12:10:03.807241 11728 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:03.784279 11214 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.811779 10582 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.858301 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.822865 10581 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
W20260709 12:10:03.866634 11810 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.005s
I20260709 12:10:03.895871 11810 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.825062 10336 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 2, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:03.872018 11831 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.898609 11831 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:03.899302 11831 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:03.865028 11214 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.905582 11831 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:03.933079 10584 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:03.940553 11791 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:03.941377 11791 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:03.945540 11791 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:03.963272 11786 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:03.982141 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 487117us
I20260709 12:10:03.990743 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:03.995158 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:04.001425 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.002640 11831 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:04.003099 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.003435 11831 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.004251 10505 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:04.005926 11831 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:04.005753 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:04.008137 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.035s	sys 0.045s
W20260709 12:10:04.027750 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.035s	sys 0.045s
W20260709 12:10:04.028164 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 782888us
I20260709 12:10:04.029649 10432 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.007190 11831 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:04.031426 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.030135 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.006230 11830 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.008735 11818 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.037030 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.036356 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.038781 10432 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:04.033733 11753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.024s	sys 0.022s
W20260709 12:10:04.041738 11753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.025s	sys 0.022s
I20260709 12:10:04.043612 10435 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:04.045069 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:04.037827 11831 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.050321 11831 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:04.052266 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.061195 11215 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:04.065390 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.066777 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.068133 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.069271 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.070272 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.071434 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.073330 10435 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.071965 11828 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:04.074887 11830 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.074615 11828 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:04.076989 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 330914us
I20260709 12:10:04.079046 11828 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:04.080590 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44320.7us
I20260709 12:10:04.081588 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:04.085814 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.087529 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.089704 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.090659 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.066372 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:04.093776 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.035054 11806 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.071888 11768 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:04.109464 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.115154 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.120729 10581 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.128262 10586 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.128142 10504 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.106310 10333 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:04.094550 11838 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.074905 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.134279 10510 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:04.094671 11803 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.094549 11824 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.124648 11822 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.105427 11830 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.097554 11728 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:04.064373 11799 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:10:04.130744 11831 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:10:04.153265 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.129374 11804 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:10:04.164665 10432 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:04.064816 11818 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:04.171900 11847 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.157886 11785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deab11eaf55b4f3ebe48926a1e2afa38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:04.111716 11806 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.155537 11822 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.178684 10504 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.189519 10513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:10:04.214375 10513 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:04.214864 10513 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:04.194700 10512 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.130091 11768 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.199172 11799 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.164093 11791 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.156220 11728 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.173482 10582 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:04.136970 11803 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.207373 11847 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.219728 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:04.225311 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45003us
I20260709 12:10:04.213536 11810 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.235149 11783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57904 (request call id 220) took 3033 ms. Trace:
I20260709 12:10:04.236479 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.238838 11790 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.211339 11829 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.242121 11803 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.245673 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:04.256752 11799 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.246549 11818 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:04.247471 11742 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.248974 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.215761 11773 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:04.253733 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.232223 11842 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:10:04.253947 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.228266 11728 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.255954 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:04.256634 10585 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.257268 11216 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:04.252367 11792 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.294315 10510 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.236992 11804 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:10:04.297355 10436 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.023s	sys 0.055s
W20260709 12:10:04.522783 10436 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.024s	sys 0.055s
I20260709 12:10:04.523288 10506 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:04.297853 10512 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:10:04.298836 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.526618 10513 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.254796 11790 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.257311 11799 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.293951 11773 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.294893 11742 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:04.423614 11806 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.425113 11824 raft_consensus.cc:993] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
W20260709 12:10:04.423149 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.032s	sys 0.016s
W20260709 12:10:04.545161 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.035s	sys 0.016s
I20260709 12:10:04.540932 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:04.549340 11806 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.550038 11806 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.526193 11830 raft_consensus.cc:476] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 12:10:04.552697 11742 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:04.425376 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 308675us
I20260709 12:10:04.565611 11806 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.567278 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:04.265537 11754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.044s	sys 0.048s
W20260709 12:10:04.571246 11754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.052s	sys 0.050s
I20260709 12:10:04.571444 11768 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.265797 11828 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.572327 11828 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.575264 10580 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 3. Ops: []
I20260709 12:10:04.576336 11801 raft_consensus.cc:2764] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Leader election decision vote started in defunct term 2: won
W20260709 12:10:04.580948 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.496099 11779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b70ca03fbe4622a03717efd079d598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:04.278831 10509 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.496973 11803 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.498651 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.499337 11847 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.499997 11829 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.588627 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.504667 10582 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:04.251010 11810 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.311551 11834 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:10:04.442552 10336 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.009s
W20260709 12:10:04.452908 11819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.000s	sys 0.017s
W20260709 12:10:04.453477 11780 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.003s	sys 0.013s
W20260709 12:10:04.453598 11802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.000s	sys 0.016s
W20260709 12:10:04.376802 11736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.008s	sys 0.023s
I20260709 12:10:04.501415 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:04.456982 11770 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.012s	sys 0.029s
I20260709 12:10:04.589998 11803 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.518839 10432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:04.624176 10432 raft_consensus.cc:606] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:04.603624 11819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.000s	sys 0.018s
W20260709 12:10:04.453162 11835 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.023s	sys 0.084s
I20260709 12:10:04.537305 10513 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:04.553748 11773 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.554957 11846 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.554754 11799 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.250696 11783 rpcz_store.cc:276] 0709 12:10:01.201865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:01.201986 (+   121us) service_pool.cc:224] Handling call
0709 12:10:01.355564 (+153578us) ts_tablet_manager.cc:934] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:10:03.207389 (+1851825us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:03.775214 (+567825us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:03.781137 (+  5923us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:04.219226 (+438089us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:04.232511 (+ 13285us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:04.234940 (+  2429us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:04.235053 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":274,"tablet-copy.queue_time_us":88465,"thread_start_us":111999,"threads_started":2}
I20260709 12:10:04.559464 11790 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.560196 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.560557 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.562256 11842 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:04.562813 11824 raft_consensus.cc:1081] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:04.564848 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:04.682806 11835 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.027s	sys 0.084s
W20260709 12:10:04.377151 11176 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.016s	sys 0.018s
I20260709 12:10:04.529568 11827 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.588207 11801 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:04.703014 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:04.590389 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:04.600464 10336 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.008s	sys 0.010s
I20260709 12:10:04.602662 11768 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.713141 11768 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.689857 11846 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.607036 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:04.715536 11768 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.715655 11846 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:04.608598 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:04.716578 10584 raft_consensus.cc:2393] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:04.610077 11780 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.004s	sys 0.014s
I20260709 12:10:04.310104 10435 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:04.589402 11828 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.632614 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:04.625818 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 124598us
I20260709 12:10:04.718744 10579 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
W20260709 12:10:04.620916 11770 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.013s	sys 0.030s
I20260709 12:10:04.621119 11834 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.722446 11834 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.686709 11819 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:04.743441 11819 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.744525 11819 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.745826 11819 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.600874 11829 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.635793 11847 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:04.618577 11802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.000s	sys 0.021s
I20260709 12:10:04.704447 11803 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.762028 11803 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:04.762521 11849 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.080s	user 0.000s	sys 0.003s
I20260709 12:10:04.763556 10431 raft_consensus.cc:3055] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:10:04.684456 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.685051 10510 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
W20260709 12:10:04.685112 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.764659 11728 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.765902 10587 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.681138 11838 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.690798 10578 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:04.567588 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.768086 11212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9042bec741aa476587972e688997da43"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57904
I20260709 12:10:04.676987 11792 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.768790 11212 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:04.769207 11212 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.769266 11802 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.769137 11792 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.769955 11802 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:04.697610 11176 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.016s	sys 0.018s
I20260709 12:10:04.674311 11865 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.770942 10435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:04.771201 11828 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.771698 10435 raft_consensus.cc:606] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:04.771500 11865 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.772258 11829 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.677167 11804 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:04.698251 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.702615 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.230726 11791 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.680325 11796 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.704319 10583 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.804432 11770 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:04.805011 11796 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.805614 11770 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.806069 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1298) took 1156 ms. Trace:
I20260709 12:10:04.806464 10583 rpcz_store.cc:276] 0709 12:10:03.649501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.650495 (+   994us) service_pool.cc:224] Handling call
0709 12:10:03.650846 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:04.805069 (+1154223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:04.805792 (+   723us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002765d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:04.805980 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422247680}
I20260709 12:10:04.705304 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:04.807783 10432 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:04.808111 10432 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:04.808643 11836 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.808862 10432 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.809146 11836 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.809878 10432 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:04.702652 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:04.699291 11783 tablet_copy_client.cc:806] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:10:04.703262 11863 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.692046 11842 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:04.706744 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:04.706391 11801 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:04.707463 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:04.680636 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47140.8us
I20260709 12:10:04.712033 11835 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.697297 11790 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.699582 11827 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.716495 11760 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.633713 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:04.638837 11810 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:04.618407 11736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.008s	sys 0.024s
I20260709 12:10:04.716037 10585 raft_consensus.cc:2393] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:04.715567 10512 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:04.728073 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1310) took 1529 ms. Trace:
W20260709 12:10:04.736332 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:04.884740 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.802616 11212 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.674919 10430 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:04.766577 11847 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.701349 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:04.791242 11728 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.854723 11806 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.792374 11829 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.777009 11838 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:04.794170 11791 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.773748 10437 raft_consensus.cc:2376] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3. Re-sending same reply.
I20260709 12:10:04.796316 11802 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:04.771708 11828 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.758492 11754 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.798503 11216 raft_consensus.cc:1240] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.773963 10431 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:04.796659 11785 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57890 (request call id 183) took 3597 ms. Trace:
I20260709 12:10:04.803339 11818 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.803845 11867 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:04.804383 11753 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.804571 11825 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.807149 11742 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:04.812326 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 109783us
I20260709 12:10:04.812983 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:04.853116 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77109.7us
I20260709 12:10:04.900732 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:04.900686 11836 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.879032 11835 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.829233 10587 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.824368 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.814775 10578 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:04.836771 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.836501 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.829736 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.826857 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:04.822490 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:04.872092 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.877220 10432 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:04.872026 10433 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:04.870245 11805 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.871310 11783 tablet_copy_client.cc:670] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:10:04.809155 11786 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.807801 11770 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:04.882843 10580 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:04.753173 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.779678 11856 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:04.851708 11842 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.852317 11846 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.890473 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.773277 11804 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.874006 11792 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:04.883886 10436 rpcz_store.cc:276] 0709 12:10:03.198759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.199417 (+   658us) service_pool.cc:224] Handling call
0709 12:10:03.199691 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:03.199902 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:03.607793 (+407891us) log.cc:844] Serialized 14 byte log entry
0709 12:10:04.727354 (+1119561us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:04.727410 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:04.727453 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:04.727521 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:04.728009 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12429,"prepare.queue_time_us":556053,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":23452,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":2122,"thread_start_us":1482944,"threads_started":3,"wal-append.queue_time_us":980969}
I20260709 12:10:04.894184 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:04.900918 10582 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:04.918515 10585 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.921109 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 219) took 1173 ms. Trace:
I20260709 12:10:04.919734 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 1313) took 1295 ms. Trace:
I20260709 12:10:04.888629 11760 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.894345 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:04.889001 11755 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.897231 11796 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.983769 10510 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:04.984524 10504 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:04.982174 10513 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:04.986469 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:05.188486 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 202064us
W20260709 12:10:05.190249 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.190011 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:04.881664 11790 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:04.914630 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.056699 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:05.059487 10584 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:05.071934 11824 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.061952 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:04.870609 11863 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.901588 11806 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.966886 11212 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.128163 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:05.133497 10512 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:04.987838 11871 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:05.115357 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 229270us
I20260709 12:10:05.178916 11792 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.135911 10430 rpcz_store.cc:276] 0709 12:10:03.613615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.613733 (+   118us) service_pool.cc:224] Handling call
0709 12:10:04.908689 (+1294956us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400436a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:04.908874 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218273152}
W20260709 12:10:05.069655 11868 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:10:05.060405 11836 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.176456 11770 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.959435 10332 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:05.055688 11865 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.969903 11825 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:10:05.200354 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 326805us
I20260709 12:10:05.207636 10584 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:05.054996 10579 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:05.073303 11768 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:05.127198 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 238564us
I20260709 12:10:05.217407 11796 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:05.213591 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:05.245100 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.959501 11780 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:10:05.137322 11791 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.198472 11755 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.960168 11754 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.216621 10431 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.244807 11802 raft_consensus.cc:993] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:04.963418 11818 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:04.969574 11753 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.240861 11760 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.271016 11753 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.211331 11736 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.271736 11802 raft_consensus.cc:1081] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:10:05.272013 11736 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.272941 11847 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:05.273458 11878 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:10:05.256197 11796 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:05.213487 11846 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:05.233228 11770 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.287662 11790 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.288358 11736 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.052977 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:05.072571 11805 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.072389 11786 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.296386 11755 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.260824 11791 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:05.300501 11874 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:10:05.254664 11835 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.920501 11785 rpcz_store.cc:276] 0709 12:10:01.198885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:01.199051 (+   166us) service_pool.cc:224] Handling call
0709 12:10:01.322653 (+123602us) ts_tablet_manager.cc:934] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
0709 12:10:04.131570 (+2808917us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:04.550251 (+418681us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:04.550395 (+   144us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:04.795861 (+245466us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:04.796198 (+   337us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:04.796523 (+   325us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:04.796591 (+    68us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":31442,"tablet-copy.queue_time_us":58880,"thread_start_us":115083,"threads_started":2}
I20260709 12:10:05.303947 11785 tablet_copy_client.cc:806] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:10:05.304986 11785 tablet_copy_client.cc:670] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:10:05.219393 11838 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.232753 11214 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:05.215579 11824 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.214640 11220 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:05.240415 11880 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.249631 11768 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.128662 11856 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.289456 11770 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.316447 11825 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
W20260709 12:10:05.073750 11876 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.193s	user 0.002s	sys 0.000s
I20260709 12:10:05.323801 11754 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.324901 11768 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:04.964497 11827 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.326345 11827 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.976325 11867 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.061029 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:05.260247 10431 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.332023 10506 raft_consensus.cc:606] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:05.221099 11792 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.300560 11791 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.223681 11871 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.267410 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:05.268440 11818 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.278067 11221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57890
I20260709 12:10:05.305874 11806 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.057245 11742 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.355933 11796 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.358371 11742 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:05.291472 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 375451us
I20260709 12:10:05.289961 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.294477 11753 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:05.295120 11863 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.274629 11804 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.225819 11836 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:05.362236 11779 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.550s	user 0.015s	sys 0.003s
I20260709 12:10:05.363685 11216 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:04.968612 11801 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.308094 10580 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:05.314579 11780 tablet_bootstrap.cc:492] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:10:05.263254 11870 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.317867 11856 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.334072 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1106) took 1123 ms. Trace:
I20260709 12:10:05.306782 11786 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.249861 11728 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.353785 11221 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:05.369104 11870 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.373874 11221 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.361209 11847 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.373896 11728 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.362356 11753 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:05.379495 11810 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.383819 11753 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.386092 11847 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.371075 10505 rpcz_store.cc:276] 0709 12:10:04.210309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.214524 (+  4215us) service_pool.cc:224] Handling call
0709 12:10:04.293731 (+ 79207us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:04.528930 (+235199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:04.538656 (+  9726us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:04.626184 (+ 87528us) log.cc:844] Serialized 26 byte log entry
0709 12:10:04.790528 (+164344us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:04.790571 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:04.790665 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:05.333305 (+542640us) raft_consensus.cc:1685] finished
0709 12:10:05.333698 (+   393us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:05.333965 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":86145,"threads_started":1,"wal-append.queue_time_us":169472}
W20260709 12:10:05.390436 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 402647us
W20260709 12:10:05.394032 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.364653 11867 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.359882 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:05.402649 11805 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.409418 11847 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.403817 11792 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.411484 11824 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.412333 11870 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.404146 11780 ts_tablet_manager.cc:1403] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 2.455s	user 0.050s	sys 0.024s
I20260709 12:10:05.416208 11779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57890 (request call id 175) took 5102 ms. Trace:
I20260709 12:10:05.417346 11856 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:05.417677 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57960.4us
I20260709 12:10:05.424213 11779 rpcz_store.cc:276] 0709 12:10:00.355072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.361645 (+  6573us) service_pool.cc:224] Handling call
0709 12:10:01.278522 (+916877us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb1a2 00007f2c988fc889 00007f2c97a2dd88 00007f2c97af1a07 00007f2c97b23f4e 00007f2c97b08fdd 00007f2c97b08f89 00007f2c97b08f19 00007f2c97b08ee1
0709 12:10:01.304210 (+ 25688us) ts_tablet_manager.cc:934] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
0709 12:10:04.291184 (+2986974us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:04.804714 (+513530us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:04.804850 (+   136us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:05.364196 (+559346us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:05.364714 (+   518us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:05.404303 (+ 39589us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:05.416060 (+ 11757us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":18998,"spinlock_wait_cycles":47220480,"tablet-copy.queue_time_us":503148,"thread_start_us":295973,"threads_started":2}
I20260709 12:10:05.429091 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:05.430684 11779 tablet_copy_client.cc:806] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:10:05.430967 11856 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:05.432394 11856 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:05.430804 11221 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.433389 11780 raft_consensus.cc:359] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:05.434427 11780 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:10:05.418821 11836 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.430696 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:05.435523 11212 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.435917 11221 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.436174 11770 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:05.434096 11856 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.469111 11780 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:05.471051 11779 tablet_copy_client.cc:670] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:10:05.484187 11871 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.222708 10332 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:05.311991 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:05.137734 10336 rpcz_store.cc:276] 0709 12:10:03.747845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.748061 (+   216us) service_pool.cc:224] Handling call
0709 12:10:03.806706 (+ 58645us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b5400270010. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb0b2 00007f2c98934759 00007f2c988cca76 00007f2c9889dd58 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc
0709 12:10:04.921024 (+1114318us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:04.562373 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:04.563029 (+   656us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:04.563071 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:10:04.563094 (+    23us) tablet.cc:662] Decoding operations
0709 12:10:04.563485 (+   391us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:04.563549 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:04.563580 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:04.563832 (+   252us) tablet.cc:701] Row locks acquired
0709 12:10:04.563858 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:10:04.564041 (+   183us) write_op.cc:270] Start()
0709 12:10:04.564192 (+   151us) write_op.cc:276] Timestamp: P: 1783599004563996 usec, L: 0
0709 12:10:04.564216 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:10:04.567477 (+  3261us) log.cc:844] Serialized 535 byte log entry
0709 12:10:04.733698 (+166221us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adae28 00007f2c96515581 00007f2c96514d58 00007f2c965199f1 00007f2c965199a9 00007f2c96519939 00007f2c96519901 00007f2c96518b2d 0000000000388634 0000000000388479
0709 12:10:04.782230 (+ 48532us) op_driver.cc:464] REPLICATION: finished
0709 12:10:04.790229 (+  7999us) write_op.cc:301] APPLY: starting
0709 12:10:04.790359 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:04.790475 (+   116us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:04.790498 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:04.790806 (+   308us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:04.791008 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:04.869703 (+ 78695us) write_op.cc:312] APPLY: finished
0709 12:10:04.889588 (+ 19885us) log.cc:844] Serialized 23 byte log entry
0709 12:10:04.902686 (+ 13098us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:04.902886 (+   200us) write_op.cc:454] Released schema lock
0709 12:10:04.903039 (+   153us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:04.905211 (+  2172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:04.909556 (+  4345us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":128186624,"child_traces":[["op",{"apply.queue_time_us":7629,"apply.run_cpu_time_us":2942,"apply.run_wall_time_us":113605,"num_ops":1,"prepare.queue_time_us":662428,"prepare.run_cpu_time_us":26509,"prepare.run_wall_time_us":171568,"raft.queue_time_us":47949,"raft.run_cpu_time_us":7899,"raft.run_wall_time_us":9820,"replication_time_us":217801,"spinlock_wait_cycles":5469440,"thread_start_us":977241,"threads_started":4,"wal-append.queue_time_us":164482}]]}
W20260709 12:10:05.526700 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 214563us
W20260709 12:10:05.530845 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:05.551071 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 120340us
I20260709 12:10:05.560473 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:05.561684 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:05.563122 10584 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
I20260709 12:10:05.598763 10579 raft_consensus.cc:2393] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:05.598371 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:05.598562 11871 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.600148 10582 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.462639 11728 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.648983 10582 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:05.671185 11870 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.688706 11870 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.691135 11792 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:05.708822 11792 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.702345 11871 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.709476 11792 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.725011 11780 ts_tablet_manager.cc:1434] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.301s	user 0.013s	sys 0.009s
W20260709 12:10:05.732113 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 555131us
I20260709 12:10:05.739880 11828 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.740320 11870 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.712560 11871 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:05.751362 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.752872 11215 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:05.756479 11770 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:05.767194 11895 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:05.767575 11770 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.768306 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.769439 11856 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:05.768051 11895 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.770238 11856 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:05.261978 11773 consensus_queue.cc:1175] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:10:05.826565 11897 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:10:05.827402 11895 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:05.834517 11792 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.837530 11792 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:05.840338 11753 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:05.840739 11842 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.013s
I20260709 12:10:05.851666 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:05.852654 11842 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.854030 11856 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:05.855677 11893 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
I20260709 12:10:05.857784 11842 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:05.858947 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.028s	sys 0.024s
W20260709 12:10:05.859381 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.031s	sys 0.024s
I20260709 12:10:05.859251 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:05.860498 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.863123 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:05.864756 11897 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:05.865244 11897 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:05.865726 11897 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.865710 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:05.867743 11894 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:05.868345 11894 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:05.869786 11894 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:05.872097 11856 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.878484 11895 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.886335 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.854938 11770 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.891701 11895 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:05.912810 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.922428 11895 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:05.924041 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.012s	sys 0.033s
W20260709 12:10:05.917384 11178 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.690s (SENT) [suppressed 78 similar messages]
W20260709 12:10:05.925412 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.013s	sys 0.034s
I20260709 12:10:05.914224 11792 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:05.934855 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.936270 11215 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:05.930964 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:05.945116 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 755206us
I20260709 12:10:05.946506 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:05.947962 11891 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:05.948443 11891 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:05.948834 11891 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.951896 11856 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.955188 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:05.956704 11885 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.960757 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.964960 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.965853 11216 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:05.975449 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.977640 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.982453 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:05.985059 11891 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.943233 11792 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:05.989533 11792 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:06.003274 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.004s	sys 0.017s
W20260709 12:10:06.003916 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 759727us
I20260709 12:10:06.006297 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:06.003980 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51817.7us
I20260709 12:10:06.013118 11870 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.014211 11885 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:06.014019 11870 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.017143 11870 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:06.017824 11870 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.022132 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.010655 11770 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.027043 11822 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.028134 11822 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:06.029994 11794 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.030432 11794 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:06.032369 10403 leader_election.cc:400] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:06.031836 11794 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.032740 10403 leader_election.cc:403] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:06.033623 11773 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.034781 11773 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:06.035970 11891 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.046569 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.050339 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:06.057181 10507 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.062144 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.057763 11897 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.071142 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:06.073722 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:06.075984 11897 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.066820 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.078061 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:06.080090 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:06.086615 10431 consensus_meta.cc:287] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.751s	user 0.016s	sys 0.012s
W20260709 12:10:06.100680 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:06.101701 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.104003 11799 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.107928 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 47) took 1406 ms. Trace:
I20260709 12:10:06.108373 11799 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.108942 11799 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.109689 11792 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.110142 11838 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.110857 11838 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.115850 11838 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.117606 11838 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.118258 11838 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:06.120621 11838 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.123855 11905 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.108510 10431 rpcz_store.cc:276] 0709 12:10:04.701890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.702047 (+   157us) service_pool.cc:224] Handling call
0709 12:10:04.702470 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:04.763444 (+ 60974us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b54002e5f20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae11c0 00007f2c92ae2231 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c
0709 12:10:05.210611 (+447167us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b4c001a8440. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c90daba20 00007f2c90dbca29 00007f2c90d82644 00007f2c90da9f2d 00007f2c90da9b93 00007f2c92ada18b 00007f2c92adacb5 00007f2c92ad5f30 00007f2c92ad77de 00007f2c92ae1b49 00007f2c92ae2416
0709 12:10:06.103733 (+893122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:06.106813 (+  3080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:06.107338 (+   525us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:06.107385 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.107501 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:06.107735 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20001024,"thread_start_us":12924,"threads_started":1}
I20260709 12:10:06.127727 11838 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.128636 11838 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.129117 11906 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:06.131153 11838 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.131678 11838 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.131759 11905 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.132717 11838 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:06.134066 10553 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.507s (SENT)
I20260709 12:10:06.137177 11838 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.138060 11838 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.139036 11888 raft_consensus.cc:993] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:10:06.138762 11838 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.139592 11888 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.140022 11888 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.141201 11888 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:06.142663 11838 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.143435 11890 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.144037 11890 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.144243 11838 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.145044 11838 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.147970 11890 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.148227 11838 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.148445 11792 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.151602 11792 raft_consensus.cc:1081] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
W20260709 12:10:06.153481 10429 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.010s	sys 0.023s
W20260709 12:10:06.154394 10429 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.011s	sys 0.023s
I20260709 12:10:06.155503 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 46) took 1454 ms. Trace:
I20260709 12:10:06.155943 10429 rpcz_store.cc:276] 0709 12:10:04.701526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.703486 (+  1960us) service_pool.cc:224] Handling call
0709 12:10:04.706875 (+  3389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:04.719821 (+ 12946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:04.719860 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:04.720466 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:10:04.906869 (+186403us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:05.213109 (+306240us) log.cc:844] Serialized 14 byte log entry
0709 12:10:06.154956 (+941847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.155171 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:06.155241 (+    70us) raft_consensus.cc:1685] finished
0709 12:10:06.155251 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:06.155438 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12455,"prepare.queue_time_us":334284,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":15182,"raft.queue_time_us":61428,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":225,"thread_start_us":1240831,"threads_started":3,"wal-append.queue_time_us":336910}
I20260709 12:10:06.159051 11799 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:06.164522 11905 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.166671 11905 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.164855 11908 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.169248 11908 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.165176 11799 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.174011 11792 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.174521 11792 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.175987 11799 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:05.558507 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:06.176842 11799 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:06.178093 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46079.9us
I20260709 12:10:06.189508 11907 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.192091 11907 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.193049 11799 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.196285 11792 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.198925 11913 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.199733 11912 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.201529 11907 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.202152 11838 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.202910 11838 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.201378 11913 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:10:06.195008 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 636766us
I20260709 12:10:06.206701 11838 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.209331 11799 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.202977 11792 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:06.215277 10401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.027s	sys 0.027s
W20260709 12:10:06.228652 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:06.231844 10401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.029s	sys 0.032s
I20260709 12:10:06.214182 11907 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.233016 11907 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.234557 11828 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:06.234781 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.237373 11911 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.237900 11911 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.238231 11910 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.239022 11911 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.239758 11799 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.240774 11911 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.240139 11847 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:06.241087 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.241371 11847 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:06.241803 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.241818 11847 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.246881 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.247679 11792 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.248096 11792 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.248230 11908 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.248739 11908 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.235142 11828 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.249162 11792 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.250118 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:06.250154 11828 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.251310 11892 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.251739 11792 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.251817 11892 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.252384 11892 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.245240 11890 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.253619 11890 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.249167 11908 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.256174 11890 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.256413 11866 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:06.257346 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.257154 11866 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.244448 11838 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.259593 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.260267 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-e4cd5d330f694dacb199bd61266c8f5a received from {username='slave'} at 127.0.0.1:36746
I20260709 12:10:06.260957 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-e4cd5d330f694dacb199bd61266c8f5a on tablet e4cd5d330f694dacb199bd61266c8f5a with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:06.262158 11892 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.263655 10511 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.264571 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:06.265328 10585 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.265640 11838 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.268555 11799 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.268810 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.269001 11889 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.269687 11889 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.269963 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.243973 11907 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.285683 10512 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:06.288514 10585 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:06.296130 11838 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.296170 11889 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.298040 11866 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.298322 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:06.300307 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:06.301419 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.303161 11913 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.303493 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.312107 11866 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.314211 11866 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:06.319175 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 82338.3us
I20260709 12:10:06.321352 11866 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.318886 11908 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.323273 11866 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:06.324110 11892 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.324887 11911 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.326004 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.339331 11212 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:06.306231 11913 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.333750 11838 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.340760 11838 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.343098 10333 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 229) took 2241 ms. Trace:
I20260709 12:10:06.335964 11866 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:06.337195 11916 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.339759 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.335371 11908 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.349005 11215 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.349412 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:06.343604 10333 rpcz_store.cc:276] 0709 12:10:04.101639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.101819 (+   180us) service_pool.cc:224] Handling call
0709 12:10:06.342672 (+2240853us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:04.734128 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:04.734739 (+   611us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:04.734775 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:10:04.734808 (+    33us) tablet.cc:662] Decoding operations
0709 12:10:04.758724 (+ 23916us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:04.759908 (+  1184us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:04.778477 (+ 18569us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:04.810379 (+ 31902us) tablet.cc:701] Row locks acquired
0709 12:10:04.810412 (+    33us) write_op.cc:260] PREPARE: finished
0709 12:10:04.810625 (+   213us) write_op.cc:270] Start()
0709 12:10:04.810805 (+   180us) write_op.cc:276] Timestamp: P: 1783599004810565 usec, L: 0
0709 12:10:04.810845 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:10:04.904220 (+ 93375us) log.cc:844] Serialized 535 byte log entry
0709 12:10:05.269119 (+364899us) op_driver.cc:464] REPLICATION: finished
0709 12:10:05.269709 (+   590us) write_op.cc:301] APPLY: starting
0709 12:10:05.269833 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:05.269989 (+   156us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:05.270125 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:05.270456 (+   331us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:05.270679 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:05.270751 (+    72us) write_op.cc:312] APPLY: finished
0709 12:10:05.271119 (+   368us) log.cc:844] Serialized 23 byte log entry
0709 12:10:05.271352 (+   233us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:05.271473 (+   121us) write_op.cc:454] Released schema lock
0709 12:10:05.271527 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:05.271952 (+   425us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:05.271990 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":348,"apply.run_cpu_time_us":2191,"apply.run_wall_time_us":2191,"num_ops":1,"prepare.queue_time_us":570400,"prepare.run_cpu_time_us":4676,"prepare.run_wall_time_us":226705,"replication_time_us":458086}]]}
I20260709 12:10:06.352036 11838 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.352648 11913 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.320297 11828 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.357025 10583 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:06.358215 11828 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.359442 11908 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.359905 11908 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.365269 11828 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:06.362711 11908 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.355636 11847 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.373179 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.374347 11212 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.361302 11866 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.376531 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:06.347625 11916 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.377954 11911 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.380422 11911 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:06.380903 11911 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.379806 11913 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.384747 11913 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.377151 11866 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.389823 11866 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.391896 11913 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.393676 11794 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.372633 11828 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.393983 11847 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.403298 11838 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.391945 11916 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.404141 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.404078 11838 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.404700 11916 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.395185 11866 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.407333 11915 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:06.408514 11838 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.409129 11915 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:06.411878 11215 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:06.415593 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.415897 11753 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.416437 10513 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.418147 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.419081 10513 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.416371 11911 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.423573 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.425414 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.426340 10513 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.428037 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.428751 10513 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.431028 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.431881 10510 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:06.433364 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.434780 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.436208 10578 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.437453 11911 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.438936 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.439848 10578 raft_consensus.cc:2368] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:06.441912 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.442263 10578 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.444581 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.445421 10510 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:06.445716 11728 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.446122 11728 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.446527 11728 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.448101 11892 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.448755 11892 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.449200 11838 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.449900 11838 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:06.452297 11838 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.452936 11892 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.454450 10505 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
W20260709 12:10:06.458348 10580 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36746 (request call id 49) took 1386 ms (client timeout 1830 ms). Trace:
I20260709 12:10:06.461522 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.462049 11914 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.462409 10578 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:06.463178 10580 rpcz_store.cc:269] 0709 12:10:05.071513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.072114 (+   601us) service_pool.cc:224] Handling call
0709 12:10:06.457884 (+1385770us) spinlock_profiling.cc:243] Waited 96 ms on lock 0x7b5400255b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:06.458095 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220741376}
I20260709 12:10:06.462517 11914 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.465479 11914 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.469563 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.476361 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.476541 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.477272 11218 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.478603 11914 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.480193 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:06.481240 11906 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.481792 11906 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.484299 11888 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.476301 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.487532 10580 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:06.493247 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.494371 11215 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:06.495101 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.495942 10513 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.496754 11218 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:06.482915 11906 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.497918 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.501827 11847 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:06.502326 11847 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.502743 11847 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.467090 11828 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:06.508083 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.509469 10580 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:06.511185 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.509018 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.518039 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.521827 10513 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.522430 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.523240 10582 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:06.524225 10578 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.506465 11822 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.527395 11728 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.536279 11728 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:06.527566 11919 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:06.538026 11919 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.541877 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.542155 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.543012 10432 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:06.543771 11919 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.543954 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.543836 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.544754 10431 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.544160 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.531474 11822 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.546464 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.547829 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.548832 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.548920 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.549597 10430 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.548012 11822 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.550405 11910 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.550661 11822 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.551645 11822 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:06.552644 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:06.553550 11728 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.553269 11822 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.553987 11728 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.554410 11728 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.551855 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.559125 11829 raft_consensus.cc:476] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 12:10:06.559201 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1122) took 1708 ms. Trace:
I20260709 12:10:06.550628 11794 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:06.550451 11810 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.562886 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.562829 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:06.569615 10580 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.522845 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.570487 10513 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.563066 10507 rpcz_store.cc:276] 0709 12:10:04.850850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.056169 (+205319us) service_pool.cc:224] Handling call
0709 12:10:06.558907 (+1502738us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54002f2bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:06.559112 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963348224}
I20260709 12:10:06.563943 11910 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.565683 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.567298 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.567723 10583 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:06.567973 10584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:10:06.574003 10584 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:06.575707 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.574118 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.579807 10584 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.575137 10430 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:06.580499 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.576232 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.508138 11913 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.581590 10508 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:06.568794 11794 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.582841 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.582939 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.580010 11221 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.587419 10580 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.577862 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.589337 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:06.590862 10504 raft_consensus.cc:2393] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.591786 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:06.582906 11794 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.526957 11212 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:06.593530 10431 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:06.593945 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.594261 10437 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.588999 11901 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.596232 11891 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.596561 11901 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:06.596704 11891 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.596903 10587 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.597168 11891 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.597648 11901 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.599022 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.599344 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.600117 11912 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.600508 11912 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.601620 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.602455 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.602829 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:06.603338 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57006.9us
I20260709 12:10:06.604513 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.605077 10508 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.606215 10586 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:06.018755 11870 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.598721 11908 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.607668 10578 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.603736 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.608897 11870 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:06.610175 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.579494 11810 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.595168 10434 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:06.612319 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.612656 11810 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.612977 11220 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.613353 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 232) took 1911 ms. Trace:
I20260709 12:10:06.613889 10335 rpcz_store.cc:276] 0709 12:10:04.678223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.678437 (+   214us) service_pool.cc:224] Handling call
0709 12:10:06.585545 (+1907108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":685650,"reported_tablets":1,"spinlock_wait_cycles":1810304}
I20260709 12:10:06.604000 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:06.616261 10509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1116) took 1767 ms (client timeout 2233 ms). Trace:
I20260709 12:10:06.616964 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:06.616698 10509 rpcz_store.cc:269] 0709 12:10:04.848996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.851904 (+  2908us) service_pool.cc:224] Handling call
0709 12:10:06.615848 (+1763944us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400152054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:06.616013 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296395136}
I20260709 12:10:06.618173 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.623984 10513 raft_consensus.cc:2410] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:06.624575 11913 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.625937 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.626844 11216 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.626276 11910 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:06.627235 11856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.006s	sys 0.000s
W20260709 12:10:06.627650 11856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.006s	sys 0.000s
I20260709 12:10:06.628291 11910 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.630254 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.631659 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.632289 10504 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:06.633221 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.633663 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:06.634347 10509 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.638239 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.638461 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.641781 10506 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:06.641925 11847 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.639932 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.642748 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.643463 10506 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.049518 11842 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:06.644307 11842 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:06.644434 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.645598 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.645419 11842 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.646301 10506 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.647017 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.643464 10505 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.647768 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:06.648543 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.649253 10505 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:06.650424 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.651535 10505 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.653862 10505 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.650885 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.654589 10506 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.656847 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:06.672035 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:06.591434 11829 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.677572 11216 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Advancing to term 2
I20260709 12:10:06.677911 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:06.679131 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74903.3us
I20260709 12:10:06.681641 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.677788 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.689013 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:06.692473 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.680157 11829 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.697639 11829 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.700644 11783 tablet_copy_client.cc:538] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:10:06.695416 11906 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.703657 11906 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:06.704761 11906 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.708065 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.708613 11847 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:06.708832 10431 raft_consensus.cc:2376] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
I20260709 12:10:06.709607 11891 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.723871 10584 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.725425 10585 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.710841 11728 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.760262 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.759984 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.771499 11891 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.759987 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.778872 10430 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.781343 11728 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.757959 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.783267 10431 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:06.784905 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:06.788035 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43496.3us
W20260709 12:10:06.770105 11926 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:10:06.797119 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:06.726765 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67234.3us
I20260709 12:10:06.759439 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.802307 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.805651 10584 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:06.807192 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62460.4us
I20260709 12:10:06.774824 11901 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:10:06.807523 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.808398 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.809406 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.812099 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.812760 10438 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:06.815589 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:06.817157 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.802079 11892 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.818493 11892 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.821138 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.821851 10430 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:06.822090 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.822948 10435 raft_consensus.cc:2368] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:06.822862 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.823529 10430 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.825011 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.825726 10509 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:06.825831 10430 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.826431 11220 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:06.826804 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.827600 10552 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.829022 11871 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.829566 11216 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:10:06.831212 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.828310 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.832490 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.833344 10438 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:06.833549 11827 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.829639 11871 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.834009 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.834151 11827 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.834658 10430 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.835878 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.836709 11847 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:06.837170 11847 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:06.837566 11847 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.835238 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:06.838129 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.838721 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.839766 11870 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.840214 11870 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.840574 11870 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.843350 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:06.843618 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:06.850482 11895 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:06.851606 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.853816 11895 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:06.805733 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:06.834183 11871 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.823966 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.855396 10552 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.859403 10432 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.844744 10434 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.863189 11932 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:06.865615 11794 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.866721 11888 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:06.867462 11822 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.868129 11907 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.868662 10429 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.759326 11912 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.871469 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.871558 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.798837 11728 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:06.887512 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.886515 10435 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.894899 11794 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.893841 11912 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.893854 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.885324 11932 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.893196 11939 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.885208 11212 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.892380 10430 raft_consensus.cc:2376] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2. Re-sending same reply.
W20260709 12:10:06.895689 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52449us
I20260709 12:10:06.894757 10438 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:06.899433 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.900748 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.900664 11822 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.901436 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.901575 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.902165 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:06.903025 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.903226 11907 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.903793 11827 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.904392 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.905079 10430 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.905467 11907 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.905974 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.906607 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:06.906651 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.908473 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.908788 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.909706 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.909855 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.907425 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.895939 11728 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.908001 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:06.910655 10434 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.913386 11728 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.914415 11810 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:06.917632 11810 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.919114 11892 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:06.919785 11810 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.898933 11932 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.919814 11218 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:06.902206 10433 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:06.916738 10429 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:06.916705 10434 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.933677 10438 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:06.936278 11783 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:10:06.939477 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 84166.7us
I20260709 12:10:06.940312 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:06.942231 10553 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.617s (SENT)
I20260709 12:10:06.945638 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.889081 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.945216 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:06.942304 11939 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:06.954882 11935 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.950441 11822 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:06.967576 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.969074 10438 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.951645 11895 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.944149 11829 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.949832 10434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:06.948410 11888 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:10:06.958863 11937 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.977706 11937 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.978181 11937 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:06.963542 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79342.8us
I20260709 12:10:06.981783 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:06.958261 11939 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:06.963939 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:06.979038 10434 raft_consensus.cc:606] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:06.985630 10431 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:06.967861 11847 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:06.979696 11828 raft_consensus.cc:2764] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:06.989389 10434 raft_consensus.cc:614] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:07.005147 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 59581.5us
I20260709 12:10:07.006673 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.007932 11892 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.008010 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.008412 11892 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.008951 11892 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.008467 11871 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:07.009294 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:07.010635 11936 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:10:07.001616 10435 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:07.003513 11810 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.011445 11910 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.968991 11870 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.012113 11910 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.015452 10437 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.016166 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:06.986670 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:07.016582 11794 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.024309  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:07.028993 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:07.019852 11842 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:07.031154 11940 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.032001 11842 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.035178 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.035859 10437 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.034626 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.023916 11871 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:07.014214 11827 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:10:07.040087 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:07.043365 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56813.9us
I20260709 12:10:07.053257 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.053929 10438 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.045856 11910 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.028867 11794 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.050045  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:10:07.041252 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 101018us
I20260709 12:10:07.028172 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.033964 11842 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.051137 11907 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:07.051810 11929 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.070394 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.073653 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:07.075814 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:07.089280 10608 debug-util.cc:398] Leaking SignalData structure 0x7b0800089900 after lost signal to thread 9264
I20260709 12:10:07.089797 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.091171  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:07.088732 11941 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:07.060603 11794 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:07.093606 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08000be520 after lost signal to thread 10370
I20260709 12:10:07.110607 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.075119 11871 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.091380 11910 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.098737 11944 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:07.059140 11829 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:07.059027 11838 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:07.111189 10608 debug-util.cc:398] Leaking SignalData structure 0x7b080013abe0 after lost signal to thread 10462
I20260709 12:10:07.085299 11907 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:07.111483 10431 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.063547 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.074468 11847 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:07.120455 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67682.1us
I20260709 12:10:07.061422 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:07.146015 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:07.140867 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08000a2560 after lost signal to thread 10537
I20260709 12:10:07.128192 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.071435 11940 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.119346 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.159510 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.160288 11940 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.160769 11940 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:07.163599 11946 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:10:07.164743 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 104) took 2350 ms. Trace:
I20260709 12:10:07.149271 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.096655 11942 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:07.147948 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08002fdd40 after lost signal to thread 10611
I20260709 12:10:07.147130 10438 raft_consensus.cc:2376] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2. Re-sending same reply.
I20260709 12:10:07.129518 11871 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.142189 11838 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.166697 11822 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:07.119292 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:07.173625 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08002a0d00 after lost signal to thread 11248
W20260709 12:10:07.174218 10608 debug-util.cc:398] Leaking SignalData structure 0x7b0800041200 after lost signal to thread 11249
W20260709 12:10:07.174069 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 95317.8us
I20260709 12:10:07.140304 11810 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:10:07.180063 10608 debug-util.cc:398] Leaking SignalData structure 0x7b0800319a60 after lost signal to thread 11789
W20260709 12:10:07.180454 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8a00 after lost signal to thread 11797
I20260709 12:10:07.146227 11907 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.144363 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.073489 11937 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.187588 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:07.177501 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 116195us
I20260709 12:10:07.191360 10433 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:07.191526 11810 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:07.192890 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:07.178797 11827 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.193668 11827 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.194049 11827 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.196775 10585 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:07.197614 10553 leader_election.cc:336] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.759s (SENT)
I20260709 12:10:07.198144 11906 raft_consensus.cc:1064] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:07.198239 10584 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:07.198860 11906 raft_consensus.cc:1068] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:07.199736 11856 raft_consensus.cc:2764] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:07.198096 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.201375 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:07.192672 11810 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.140079 11910 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.172447 11942 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.171674 11870 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:07.183919 10429 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.170183 11212 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:07.179268 11838 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:07.206689 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57527.7us
I20260709 12:10:07.179498 11822 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:07.165139 10332 rpcz_store.cc:276] 0709 12:10:04.807449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.807653 (+   204us) service_pool.cc:224] Handling call
0709 12:10:07.157482 (+2349829us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:06.472883 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:06.473686 (+   803us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:06.473735 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:10:06.473755 (+    20us) tablet.cc:662] Decoding operations
0709 12:10:06.474246 (+   491us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:06.474360 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:06.474396 (+    36us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:10:06.474927 (+   531us) tablet.cc:701] Row locks acquired
0709 12:10:06.475026 (+    99us) write_op.cc:260] PREPARE: finished
0709 12:10:06.475216 (+   190us) write_op.cc:270] Start()
0709 12:10:06.475378 (+   162us) write_op.cc:276] Timestamp: P: 1783599006475154 usec, L: 0
0709 12:10:06.475422 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:10:06.514536 (+ 39114us) log.cc:844] Serialized 907 byte log entry
0709 12:10:06.866418 (+351882us) op_driver.cc:464] REPLICATION: finished
0709 12:10:06.986352 (+119934us) write_op.cc:301] APPLY: starting
0709 12:10:06.986488 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:06.986653 (+   165us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:06.988618 (+  1965us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:07.023699 (+ 35081us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:07.063172 (+ 39473us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:10:07.091570 (+ 28398us) write_op.cc:312] APPLY: finished
0709 12:10:07.102111 (+ 10541us) log.cc:844] Serialized 29 byte log entry
0709 12:10:07.102355 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:07.102470 (+   115us) write_op.cc:454] Released schema lock
0709 12:10:07.102544 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:07.103058 (+   514us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:07.103093 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":923903,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":105017,"apply.run_cpu_time_us":3533,"apply.run_wall_time_us":119438,"num_ops":2,"prepare.queue_time_us":14271,"prepare.run_cpu_time_us":5177,"prepare.run_wall_time_us":43218,"raft.queue_time_us":229731,"raft.run_cpu_time_us":21267,"raft.run_wall_time_us":75723,"replication_time_us":390812,"thread_start_us":160724,"threads_started":4,"wal-append.queue_time_us":1398}]]}
I20260709 12:10:07.218884 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.227957 10437 raft_consensus.cc:2376] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2. Re-sending same reply.
I20260709 12:10:07.219446 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.228328 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.219568 11942 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.219746 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.232136 10431 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.183252 10335 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 3, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:07.220530 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.233644 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.221292 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.187789 11929 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:07.226961 11847 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.247373 11929 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:07.248678 11871 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:07.248587 11929 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.231160 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.214159 11220 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.221997 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.240656 10335 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:07.219509 11910 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.244433 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.245273 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.246191 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.259856 10432 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:07.260164 10431 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.260277 10432 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:07.253199 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.261595 10432 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.230264 11822 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.265190 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.266448 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.269129 10432 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.269193 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.274786 10438 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.276018 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.276088 10335 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:07.282428 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.280771 10335 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:07.285146 11950 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.286378 11950 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.286757 11950 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.289860 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.284636 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.317477 10438 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:07.321041 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.324512 11753 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:07.278689 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.284955 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.287827 11937 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:07.280525 11728 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:07.335640 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51092.3us
I20260709 12:10:07.362739 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.368402 10430 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
W20260709 12:10:07.342815 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114567us
W20260709 12:10:07.345064 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 269397us
I20260709 12:10:07.363524 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.373987 10431 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:07.284554 10335 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:07.375485 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.379356 11932 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.380609 11827 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.338732 10435 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.339805 10434 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.390712 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.339784 11906 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:07.400315 10579 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:07.331125 11935 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 12:10:07.409582 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.433528 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.433952 10587 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:07.350929 11822 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.435961 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.444051 10335 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:07.438258 10432 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:07.438836 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.442480 10586 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.443221 11810 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:07.435207 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.448895 10583 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:07.441228 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65880.7us
I20260709 12:10:07.377048 11907 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.462373 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.456102 10335 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:07.442076 10429 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:07.436158 11940 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.448756 11906 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:07.442121 11838 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.442400 11842 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.475494 11212 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:07.461560 11810 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:07.488647 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.495501 10581 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:07.486889 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:07.502059 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.507686 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:07.496155 11912 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.007s
I20260709 12:10:07.497454 11810 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.515396 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.516567 10432 raft_consensus.cc:2368] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3.
W20260709 12:10:07.517670 11963 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:10:07.521163 11822 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.514698 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.521920 11822 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.522677 11822 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.523128 11822 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.529750 10436 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:10:07.539465 11956 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:10:07.495887 11220 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:07.542475 11842 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:07.544242 11932 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:07.545984 11827 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.547871 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:07.548352 11838 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.549333 11828 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:07.549249 11906 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.550109 11828 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:07.551287 11940 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:07.511832 11892 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.005s
I20260709 12:10:07.552605 11892 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:07.548177 11907 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.554845 11892 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:07.552208 11828 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:07.563604 10437 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:07.569942 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1335) took 1029 ms. Trace:
I20260709 12:10:07.571203 11961 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:07.572199 11961 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:07.572513 11961 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:07.576128 10438 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:07.577889 11728 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:07.580114 10585 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:07.570602 10436 rpcz_store.cc:276] 0709 12:10:06.540780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.853018 (+312238us) service_pool.cc:224] Handling call
0709 12:10:06.853341 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:07.530211 (+676870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:07.569532 (+ 39321us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54002661d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:07.569818 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514819840}
I20260709 12:10:07.587992 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:07.589339 11212 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.599033 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.562490 11950 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:07.603901 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 141631us
I20260709 12:10:07.566496 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:07.605400 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.605871 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.615195 11907 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:10:07.620476 11907 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:07.620901 11907 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:07.621763 11907 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.604597 11794 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.625308 11794 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:07.627405 11178 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.690s (SENT)
I20260709 12:10:07.629864 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.628954 11838 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:10:07.612226 11957 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:07.638137 10435 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:07.639235 11827 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:07.641741 10584 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:07.644240 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.645547 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.645946 10582 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:07.574244 11961 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:07.651911 10553 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.501s (SENT)
I20260709 12:10:07.664196 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.668208 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.669040 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.675786 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:07.679190 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:07.680047 10508 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4.
I20260709 12:10:07.680706 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:07.631676 11907 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.683931 10583 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:07.684216 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:07.685715 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:07.686370 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.687381 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.688436 11950 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.689587 10584 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:07.691397 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:07.685264 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:07.695714 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:07.667621 10429 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:07.700953 11895 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:07.701601 11961 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.702610 10509 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:07.702391 11895 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:07.707765 10434 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:07.688453 10582 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:07.732296 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.732719 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.700981 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.684944 11940 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:07.734663 10435 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.735692 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.736747 10582 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:07.739683 11221 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:07.733824 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.740509 10434 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:07.711052 11953 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.266s	user 0.006s	sys 0.000s
I20260709 12:10:07.712425 11935 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:07.685344 10513 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:10:07.716403 10584 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:07.719309 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.720492 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.720796 11810 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.720971 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:07.720544 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:07.721297 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.721900 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.722198 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.723203 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.723873 11942 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.703781 10510 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:07.725019 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:07.687515 11964 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:10:07.727035 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.727948 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.759387 10430 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.730500 10431 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:07.753593 10581 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:07.755817 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:07.756956 11178 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.019s (SENT)
I20260709 12:10:07.765556 10586 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:07.772462 10587 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.772653 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.777179 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.778956 10583 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:07.780472 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.742408 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:07.782032 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 130177us
I20260709 12:10:07.788818 10580 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.785045 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.791999 10578 raft_consensus.cc:2368] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4.
I20260709 12:10:07.774627 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:07.794062 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:07.800083 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.802038 10508 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:07.802480 10581 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.804936 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.805859 10511 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:07.807235 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.785861 11783 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:10:07.810030 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.811360 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:07.821488 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64593.5us
I20260709 12:10:07.823778 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.811538 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.812409 10513 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:07.826084 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.827742 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:07.828622 10507 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:07.829789 10583 raft_consensus.cc:2393] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.815892 10587 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:07.832855 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:07.835475 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.836387 10438 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.806559 10429 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.835506 10430 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:07.839421 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.842636 11178 leader_election.cc:400] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3.
I20260709 12:10:07.843883 10585 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.850219 10580 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:07.827275 11220 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:10:07.859326 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.861748 10584 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:07.861727 11178 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:07.867481 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56053.7us
I20260709 12:10:07.852387 10582 raft_consensus.cc:2368] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:07.868798 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.871080 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:07.870505 10579 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.829676 11929 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:07.874800 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:07.840866 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:07.885262 10507 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:07.887858 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.898082 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.869035 11935 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.896180 11906 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.876811 11888 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.879997 11962 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.928196 11962 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.880012 11930 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:10:07.932348 11940 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:07.890465 11838 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:07.935364 11838 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:07.936439 11838 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.939894 11810 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.941473 11779 tablet_copy_client.cc:538] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:10:07.942078 11822 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:07.942821 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.944388 11822 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.944968 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.945782 11822 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:07.946395 11822 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.948782 11822 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:07.948626 11957 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:07.949273 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:07.925153 11785 tablet_copy_client.cc:538] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:10:07.927115 11888 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.856112 11828 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.872195 11932 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:10:07.889370 11965 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:07.859700 11950 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 12:10:07.956218 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 87524.6us
W20260709 12:10:07.957031 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 298004us
I20260709 12:10:07.960070 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:07.921998 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:07.945797 10437 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.967582 11783 tablet_bootstrap.cc:492] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:10:07.973708 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:07.901849 10509 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:07.976411 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:07.998553 10587 raft_consensus.cc:2368] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:07.965018 11912 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.000734 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.005347 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:08.006425 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.007119 10580 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.008924 10578 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:07.972860 10433 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:08.013494 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:07.968199 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:08.018672 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.969223 11212 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:08.016954 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48922.9us
I20260709 12:10:07.961795 11906 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:07.967960 11930 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:10:08.026760 10437 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:08.028400 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.028980 11948 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:08.038713 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.043514 10578 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:08.038590 10580 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:08.040494 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.022784 11827 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:10:08.046926 11888 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.047209 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:08.016965 11912 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.047863 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.048569 10579 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.049916 11912 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.059792 10585 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.020362 11965 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.037336 11783 ts_tablet_manager.cc:1403] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 1.101s	user 0.028s	sys 0.014s
I20260709 12:10:08.062357 10587 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:08.077939 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:08.079846 10438 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.053228 11220 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:08.087778 11940 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.088632 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:08.096091 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63116.7us
I20260709 12:10:08.060135 11912 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:08.067072 11937 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:08.097108 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.099164 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.058213 11962 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.105109 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.107661 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:08.110638 10587 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.119361 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.126929 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.082048 11942 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:08.135565 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.136435 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:08.138760 10506 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:08.162686 10585 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:08.168097 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:08.087260 11965 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:08.148237 10512 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:08.164889 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.175630 10513 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.165235 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.176671 10504 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:08.164979 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.136133 11822 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.177449 10511 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.109267 11937 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:08.189177 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.190219 10508 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:08.135938 11957 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:10:08.200488 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:08.213548 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.137185 11912 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.214502 10504 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.185755 11937 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.220180 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:08.200495 11907 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 12:10:08.202239 11948 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.224319 10512 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.184358 11940 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.179997 11888 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:08.186026 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.136996 11895 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.254534 10438 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
W20260709 12:10:08.227488 11970 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:10:08.239989 11779 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:10:08.242957 11974 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 12:10:08.282647 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 185644us
W20260709 12:10:08.283639 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 196992us
W20260709 12:10:08.287078 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 123655us
I20260709 12:10:08.248361 10582 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:08.227900 11838 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:10:08.314060 10579 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:08.314440 11935 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:08.240183 11982 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:10:08.311370 11957 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.315701 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.315625 11957 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.316401 10509 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:08.317579 10435 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.006s
I20260709 12:10:08.315021 11935 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:08.318262 11770 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:08.319213 11770 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:08.320107 11770 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:08.319510 11935 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.321004 11770 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:08.321597 11935 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.321933 11929 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:08.322343 11929 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:08.300729 11783 raft_consensus.cc:359] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:08.246493 11888 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.323263 11783 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:10:08.323695 10511 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:10:08.303093 11977 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:10:08.281363 11930 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.251791 11907 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.263073 11913 raft_consensus.cc:2764] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:10:08.320382 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.324179 11783 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:08.325032 11888 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.326663 11982 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:08.285832 11906 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.303730 11965 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.294525 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:08.280009 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.331422 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:08.328567 11961 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.626s	user 0.008s	sys 0.004s
I20260709 12:10:08.330862 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.352957 10504 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:08.363561 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.364246 10504 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:08.371260 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.376518 10432 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:08.317970 10435 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:08.381455 11961 raft_consensus.cc:2764] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:10:08.333405 11822 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:08.383955 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 89568.7us
I20260709 12:10:08.384939 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.386272 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.327610 10509 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:08.379715 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 99805.8us
I20260709 12:10:08.370761 10587 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:08.399006 11785 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:10:08.387894 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.318004 11957 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.364199 11982 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:08.402671 10509 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:08.405248 11828 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:08.407593 11987 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:08.412168 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:08.380448 11986 raft_consensus.cc:2764] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 12:10:08.406795 10434 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.006s
I20260709 12:10:08.415318 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:08.417138 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 86357.2us
I20260709 12:10:08.421883 11218 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:08.372400 11930 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.435158 11930 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:08.429703 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.422305 10436 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:08.412436 11906 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:08.437247 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.431806 10434 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:08.432180 11783 ts_tablet_manager.cc:1434] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.344s	user 0.009s	sys 0.001s
I20260709 12:10:08.379103 11948 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.433001 11828 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.362190 11907 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.438552 10504 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:08.438613 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.407744 11940 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.413976 11827 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:08.440618 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50395.4us
I20260709 12:10:08.447844 11220 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:08.448400 11982 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.451637 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.439700 11906 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:08.452584 11906 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.453507 10508 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:08.456435 10513 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:08.452291 11822 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.430827 11987 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.457183 11910 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:08.459030 11987 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.436880 11912 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.462131 11907 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:08.464346 11982 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.453828 11942 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.475011 11990 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:08.378820 11929 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.477273 11929 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:08.486323 11937 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:08.494926 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52250.2us
I20260709 12:10:08.498342 10512 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:08.499235 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:08.499526 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62363.8us
W20260709 12:10:08.500942 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:08.429040 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 111524us
I20260709 12:10:08.504179 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:08.503379 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:08.451265 11985 raft_consensus.cc:2764] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:10:08.495899 11995 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:10:08.498571 11961 raft_consensus.cc:2764] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:10:08.538656 11994 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:08.543987 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.508829 11910 consensus_queue.cc:1059] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:10:08.548220 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:08.549342 12003 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:08.555503 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:08.558048 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.565759 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:08.533504 11962 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:10:08.558633 11942 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:08.561491 11986 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.576596 10403 leader_election.cc:400] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4.
I20260709 12:10:08.567071 11985 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.561134 12003 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.579712 11937 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:08.570462 11912 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:08.567685 11910 consensus_queue.cc:1243] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:08.573779 11895 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.587762 11985 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:08.586225 11961 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:08.588547 11983 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.589965 12003 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.597921 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:08.614462 11828 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.615227 11828 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.616447 11828 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.605233 11942 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.610352 11910 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:08.629128 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:08.630498 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:08.633002 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:08.632037 11910 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:08.639654 11937 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.608536 11983 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.647660 11948 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.670863 11985 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:08.674700 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 171436us
I20260709 12:10:08.644399 11986 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.677982 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:08.641651 12004 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 12:10:08.678193 12001 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.115s	user 0.000s	sys 0.002s
I20260709 12:10:08.675750 11942 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.674193 11937 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.696673 11906 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.699436 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1384) took 1023 ms. Trace:
I20260709 12:10:08.700109 11906 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.699975 10429 rpcz_store.cc:276] 0709 12:10:07.675681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.899391 (+223710us) service_pool.cc:224] Handling call
0709 12:10:07.968823 (+ 69432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:08.178888 (+210065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:08.188816 (+  9928us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:08.251062 (+ 62246us) log.cc:844] Serialized 26 byte log entry
0709 12:10:08.416984 (+165922us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:08.417051 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:08.465978 (+ 48927us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b1800294220. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:08.699069 (+233091us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b54002661d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:08.699139 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:08.699193 (+    54us) raft_consensus.cc:1685] finished
0709 12:10:08.699206 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:08.699363 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44753,"raft.queue_time_us":55882,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":24383,"spinlock_wait_cycles":209561344,"thread_start_us":179165,"threads_started":2,"wal-append.queue_time_us":241751}
I20260709 12:10:08.681092 11910 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:08.716074 11728 raft_consensus.cc:2764] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:10:08.729398 11962 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.730345 11962 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:08.735340 11962 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.736286 11983 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.723417 11912 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:08.742435 11912 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.745573 11912 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:08.690147 12011 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.751626 12011 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.761085 12011 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.764590 11982 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:08.765300 11982 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:08.785054 11827 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:08.785557 11996 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.767606 11975 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:08.797564 11975 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:08.799073 11975 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.801031 11975 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:08.802484 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 354880us
I20260709 12:10:08.802845 10507 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:08.788197 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 349655us
I20260709 12:10:08.787014 11827 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.776517 11948 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.766705 11982 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.717157 11937 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.806461 11913 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.808177 10512 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:08.804877 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:08.829551 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.831712 12014 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:10:08.833766 12014 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:08.838064 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:08.824883 11965 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:08.839314 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:08.839567 11728 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:08.840198 11728 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.823698 11948 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.830776 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.840749 11991 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:08.852291 11982 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.839646 11965 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.855706 11981 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:10:08.837066 12014 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:08.867895 11989 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.869343 11989 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:08.872752 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.511s (SENT)
I20260709 12:10:08.848600 11996 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:08.823714 11913 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.770123 11986 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:08.880702 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 303149us
I20260709 12:10:08.900286 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:08.863704 11991 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.893906 11996 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:08.902760 11907 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.905421 11991 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.911098 10403 leader_election.cc:400] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:08.906473 11985 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:08.921025 11985 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.921603 11985 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:08.899672 11728 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.915339 11996 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.924328 10403 leader_election.cc:403] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:08.923635 11907 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:08.927954 12014 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.925429 11785 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:10:08.930178 11913 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:08.930708 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.932127 11785 tablet_bootstrap.cc:492] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:10:08.931952 11989 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:08.930269 12014 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:08.939440 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.939919 11785 ts_tablet_manager.cc:1403] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.541s	user 0.036s	sys 0.019s
I20260709 12:10:08.942549 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:08.943595 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.914449 11965 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:08.946908 11962 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:08.948109 11962 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:08.948674 11962 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:08.928086 11907 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.949268 11962 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.956455 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:08.959192 11961 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:08.959928 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:08.960026 11961 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:08.961800 11961 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:08.963550 11888 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:08.961815 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:08.957490 11913 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:10:09.004060 11929 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:09.007754 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46016.4us
I20260709 12:10:08.996506 11888 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.023694 11913 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.005179 11929 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.010821 11985 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:09.049226 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 371286us
W20260709 12:10:09.076972 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.083688 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.084771 11985 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:09.085125 11940 raft_consensus.cc:2764] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:09.085036 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.086727 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.087469 10507 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:09.082669 11929 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:09.081681 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.088611 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.089807 11888 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.092375 12002 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:09.093921 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.094693 11910 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.095008 12002 raft_consensus.cc:2764] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:09.096865 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.100668 12002 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.101285 12002 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:09.103410 11942 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.102423 12002 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:09.089887 11981 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.105937 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.103545 11912 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:09.112838 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:09.123870 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.138859  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:09.079859 11913 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.156343 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:09.159600 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 318860us
I20260709 12:10:09.161517 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.137358 11930 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:09.177387 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71540.1us
I20260709 12:10:09.180935 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.121640 11912 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.180403 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:09.153172 11935 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:09.189430 11979 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:09.190152 11979 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:09.190766 11979 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:09.186230 11965 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:09.194478 11912 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:09.170305 11975 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:09.165593 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45648us
I20260709 12:10:09.159176 11982 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:10:09.205644 11218 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:09.180996 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.188700 11942 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.169206 11838 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:10:09.194780 11728 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:09.197602 11935 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.203305 11965 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.224543 10510 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:09.206504 11976 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:10:09.185709 11930 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.210892 12017 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:09.217228 11906 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:09.241310 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60386.4us
I20260709 12:10:09.229199 11810 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:09.239130 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 408419us
I20260709 12:10:09.228799 11965 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:09.225842 11942 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:09.217506 11991 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.243009 11982 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.241568 11906 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:09.266086 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.292191 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.290548 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:09.322270 10505 raft_consensus.cc:2368] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3.
I20260709 12:10:09.262360 12023 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:10:09.323760 11942 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.243651 12017 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.327032 11913 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:09.290894 11982 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.330005 12017 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:09.288259 11948 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:10:09.337371 11991 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:09.338665 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46545.4us
I20260709 12:10:09.339720 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.341476 11957 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:09.346611 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:09.342566 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:09.354200 10553 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.649s (SENT)
W20260709 12:10:09.312392 12020 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 12:10:09.355418 10553 leader_election.cc:336] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.660s (SENT)
I20260709 12:10:09.360888 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.287695 11906 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:09.365782 10553 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.750s (SENT)
W20260709 12:10:09.366957 10553 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.771s (SENT)
I20260709 12:10:09.371677 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:09.311345 11810 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:09.260253 11988 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:09.373517 11948 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.378422 12027 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:09.385272 12022 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.159s	user 0.000s	sys 0.002s
I20260709 12:10:09.369467 12014 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.385319 11948 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.373606 11838 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
W20260709 12:10:09.389456 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.374502 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:09.377198 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:09.373955 11907 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.398934 11948 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:09.372570 11991 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:09.411803 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:09.408967 12014 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.413321 11965 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.413924 11975 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:09.413498 12014 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:09.414484 11975 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:09.383180 11810 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.394784 11982 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:09.319768 11785 raft_consensus.cc:359] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:09.448199 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 166394us
I20260709 12:10:09.448508 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:09.448961 10553 leader_election.cc:336] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.822s (SENT)
I20260709 12:10:09.449411 10504 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:10:09.451488 10553 leader_election.cc:336] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.839s (SENT)
W20260709 12:10:09.455749 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 81361.3us
I20260709 12:10:09.456815 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.458354 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.431699 11930 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:09.498414 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.465206 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:09.476495 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:09.499346 10430 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:10:09.516883 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65420.6us
W20260709 12:10:09.531787 10553 leader_election.cc:336] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.899s (SENT)
I20260709 12:10:09.532342 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:09.526199 11218 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:09.536993 10507 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:09.546193 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:09.552407 10435 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:10:09.487205 11962 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.003s
W20260709 12:10:09.476622 12026 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.060s	user 0.000s	sys 0.002s
I20260709 12:10:09.494956 11987 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:09.500511 11996 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.007s
I20260709 12:10:09.446760 11785 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
W20260709 12:10:09.562618 10553 leader_election.cc:336] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.919s (SENT)
I20260709 12:10:09.564553 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:09.564735 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:09.573431 10431 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.445959 11810 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.465512 11991 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.508497 12014 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:09.478133 11212 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:10:09.572232 11996 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:09.619414 11212 pending_rounds.cc:85] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 4
I20260709 12:10:09.499720 11838 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.486119 11935 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:09.393275 11988 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.599155 10552 leader_election.cc:400] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3.
I20260709 12:10:09.561487 11981 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:09.572898 11987 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.569479 11962 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:09.650235 10552 leader_election.cc:403] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:09.660888 12017 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.480301 11975 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:09.780161 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:10.140627 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.863623 10431 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
W20260709 12:10:09.970721 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.026s	sys 0.044s
W20260709 12:10:10.145394 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.034s	sys 0.048s
W20260709 12:10:10.146373 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 965459us
W20260709 12:10:10.137552 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 574930us
I20260709 12:10:10.155505 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:09.765792 10512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.026s	sys 0.043s
W20260709 12:10:10.160019 10512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.220s	user 0.027s	sys 0.065s
W20260709 12:10:09.802568 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.020s	sys 0.040s
W20260709 12:10:10.163399 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.024s	sys 0.056s
W20260709 12:10:09.700781 12011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.020s	sys 0.044s
W20260709 12:10:10.194530 12011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.028s	sys 0.052s
W20260709 12:10:09.928668 11728 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.008s
I20260709 12:10:10.195060 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:10.195526 12011 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:10.195742 11981 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.196501 11981 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:10.196403 12011 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:09.619917 11212 pending_rounds.cc:107] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:10:10.222365 10436 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:10.218966 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:09.771159 11979 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.005s
W20260709 12:10:09.774166 11895 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.005s	sys 0.018s
W20260709 12:10:09.855484 11779 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.025s	sys 0.036s
W20260709 12:10:09.963721 11989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.028s	sys 0.055s
W20260709 12:10:09.942757 11910 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.847s	user 0.012s	sys 0.010s
W20260709 12:10:10.270892 12002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.013s	sys 0.017s
W20260709 12:10:10.325815 11178 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.084s	user 0.010s	sys 0.025s
W20260709 12:10:09.929352 11984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.017s	sys 0.048s
W20260709 12:10:10.332226 10430 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.007s	sys 0.012s
W20260709 12:10:10.407475 10430 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.009s	sys 0.013s
W20260709 12:10:10.025264 12031 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.104s	user 0.004s	sys 0.000s
W20260709 12:10:09.762853 11986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.034s	sys 0.042s
W20260709 12:10:09.993778 11957 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.008s
I20260709 12:10:09.736948 12008 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:09.766418 11991 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.145123 11987 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.264962 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:10.457043 10437 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:10.341544 11827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.015s	sys 0.014s
I20260709 12:10:09.674039 11930 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.137012 11996 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.495618 11996 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.496277 11996 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:10.470657 11827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.016s	sys 0.014s
W20260709 12:10:10.273279 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.004s
W20260709 12:10:10.350692 11178 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.011s	sys 0.025s
W20260709 12:10:10.318018 11961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.122s	user 0.004s	sys 0.018s
W20260709 12:10:10.324709 11906 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.942s	user 0.009s	sys 0.008s
I20260709 12:10:10.304929 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:10.513908 10429 raft_consensus.cc:2376] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
I20260709 12:10:10.515763 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:10.517066 10429 raft_consensus.cc:2368] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3.
I20260709 12:10:10.524742 11965 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.532768 11988 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.533671 11981 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:10.535655 11985 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.536516 11985 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:10.539881 11985 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.540640 11985 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.540751 11975 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:10.541255 11985 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:10.544472 11985 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:10.548266 12027 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.162s	user 0.016s	sys 0.002s
I20260709 12:10:10.549014 12027 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.265247 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:10.311058 11907 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.002s	sys 0.017s
W20260709 12:10:10.334138 11779 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.399s	user 0.026s	sys 0.045s
I20260709 12:10:10.268543 11929 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.325619 11212 raft_consensus.cc:2889] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:10.581972 11932 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.364s	user 0.039s	sys 0.023s
W20260709 12:10:10.582790 11932 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.388s	user 0.039s	sys 0.023s
I20260709 12:10:10.583523 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 248) took 1143 ms. Trace:
I20260709 12:10:10.582283 11929 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:10.584391 11212 rpcz_store.cc:276] 0709 12:10:09.440204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.441171 (+   967us) service_pool.cc:224] Handling call
0709 12:10:09.477458 (+ 36287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:10.582840 (+1105382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:10.583354 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:10.460518 12008 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:10.587698 11932 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:10.335668 11895 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.012s	sys 0.025s
I20260709 12:10:09.836273 11935 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:10.325433 11822 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.014s	sys 0.019s
W20260709 12:10:10.327386 11770 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.008s	sys 0.026s
I20260709 12:10:10.335975 11979 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:10.351697 12002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.017s	sys 0.017s
W20260709 12:10:10.339725 11989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.307s	user 0.029s	sys 0.072s
I20260709 12:10:10.609452 11932 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:10.617601 11932 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.618126 11929 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:10.618412 11932 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:10.416070 11986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.461s	user 0.050s	sys 0.056s
W20260709 12:10:10.437292 11984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.313s	user 0.024s	sys 0.061s
I20260709 12:10:10.555594 12038 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:10.612236 11770 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.404s	user 0.009s	sys 0.027s
I20260709 12:10:10.636087 11979 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:10.620538 11935 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:10.640446 11935 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.644529 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1402) took 1100 ms. Trace:
I20260709 12:10:10.644953 10436 rpcz_store.cc:276] 0709 12:10:09.544309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.544479 (+   170us) service_pool.cc:224] Handling call
0709 12:10:09.544969 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:10.308281 (+763312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:10.644230 (+335949us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b54003a8ed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:10.644452 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557190016}
I20260709 12:10:10.200556 12017 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:10.494045 11930 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:10.500674 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 1.2688e+06us
W20260709 12:10:10.656458 11178 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.754s (SENT)
W20260709 12:10:10.500291 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.003s	sys 0.008s
I20260709 12:10:10.511610 11996 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:10.511739 11961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.323s	user 0.004s	sys 0.022s
I20260709 12:10:10.512435 11906 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:10.692308 10432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.132s	user 0.009s	sys 0.022s
I20260709 12:10:10.518460 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:10.578693 11962 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:10.197860 11728 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:10.600672 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:10.748150 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:10.604221 10513 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:10:10.568418 11907 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.003s	sys 0.018s
I20260709 12:10:10.443946 11957 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.595536 10433 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.421531 11910 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:10.609884 11822 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.273s	user 0.014s	sys 0.020s
I20260709 12:10:10.461313 11991 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:10.657804 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:10.667800 10437 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:10.670135 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1170) took 1719 ms. Trace:
I20260709 12:10:10.602728 12008 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.688026 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:10.238279 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:10.657862 11178 connection.cc:256] RPC call timeout handler was delayed by -1.2156584110000495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 12:10:10.198680 11785 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:10.627411 11932 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:10.724249 10432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.009s	sys 0.023s
W20260709 12:10:10.745028 10510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.344s	user 0.007s	sys 0.033s
I20260709 12:10:10.689939 11984 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:10.652474 12038 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:10.783175 10435 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:10.783851 12011 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:10.782491 10429 raft_consensus.cc:2376] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
W20260709 12:10:11.372454 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.002s	sys 0.011s
I20260709 12:10:11.406786 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:10.995328 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.536s	user 0.025s	sys 0.023s
W20260709 12:10:11.413576 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.954s	user 0.034s	sys 0.031s
W20260709 12:10:11.414152 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:11.414331 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 1.95594e+06us
I20260709 12:10:11.414516 12060 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.415248 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:11.415064 12060 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.416632 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:11.372828 11770 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.004s	sys 0.008s
I20260709 12:10:11.417233 12060 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:11.417414 11770 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.004s	sys 0.008s
I20260709 12:10:11.417783 12060 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:11.417975 10478 leader_election.cc:400] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
W20260709 12:10:11.166940 10553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.018s	sys 0.041s
W20260709 12:10:11.418478 10553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.020s	sys 0.041s
I20260709 12:10:11.418288 12060 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:11.314003 11987 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.003s
I20260709 12:10:11.419263 10434 raft_consensus.cc:2376] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
I20260709 12:10:11.419966 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1119) took 1271 ms. Trace:
I20260709 12:10:11.128515 11906 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:10.931452 11942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.590s	user 0.034s	sys 0.049s
I20260709 12:10:11.421077 12060 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:11.223683 11827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.003s	sys 0.022s
W20260709 12:10:11.160125 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.027s	sys 0.030s
W20260709 12:10:11.458920 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.294s	user 0.029s	sys 0.036s
W20260709 12:10:10.857056 10585 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.499s	user 0.012s	sys 0.065s
W20260709 12:10:11.459985 10585 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.102s	user 0.015s	sys 0.080s
I20260709 12:10:10.805295 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1175) took 1403 ms. Trace:
I20260709 12:10:11.413517 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1121) took 1255 ms. Trace:
W20260709 12:10:11.412981 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.003s	sys 0.011s
I20260709 12:10:11.423748 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1409) took 1864 ms. Trace:
W20260709 12:10:11.421490 11942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.080s	user 0.047s	sys 0.058s
W20260709 12:10:11.023437 10552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.006s	sys 0.038s
W20260709 12:10:11.468318 10552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.017s	sys 0.043s
I20260709 12:10:11.461277 10432 rpcz_store.cc:276] 0709 12:10:09.559000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.559170 (+   170us) service_pool.cc:224] Handling call
0709 12:10:09.559553 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:09.559811 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:09.560185 (+   374us) log.cc:844] Serialized 14 byte log entry
0709 12:10:11.423370 (+1863185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:11.423431 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:11.423485 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.423559 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.423675 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1662,"prepare.queue_time_us":729,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":7534,"thread_start_us":1862830,"threads_started":1,"wal-append.queue_time_us":1554995}
I20260709 12:10:11.461763 10512 rpcz_store.cc:276] 0709 12:10:08.937634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.937880 (+   246us) service_pool.cc:224] Handling call
0709 12:10:08.938397 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:08.938933 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:08.938963 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:08.939694 (+   731us) log.cc:844] Serialized 26 byte log entry
0709 12:10:10.160566 (+1220872us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:10.160634 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:10.160738 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:10.656731 (+495993us) raft_consensus.cc:1685] finished
0709 12:10:10.656759 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:10.656949 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1545,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":4771,"thread_start_us":1220331,"threads_started":1,"wal-append.queue_time_us":1679715}
I20260709 12:10:11.416738 11932 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:10.885035 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.443s	user 0.020s	sys 0.059s
W20260709 12:10:11.362701 10430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.007s	sys 0.016s
W20260709 12:10:11.365121 11975 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.824s	user 0.013s	sys 0.004s
W20260709 12:10:11.472254 10430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.009s	sys 0.016s
I20260709 12:10:11.439320 10478 leader_election.cc:400] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
W20260709 12:10:11.472172 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.031s	user 0.032s	sys 0.071s
W20260709 12:10:11.473157 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54284.2us
I20260709 12:10:11.473930 10478 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:11.067991 11838 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.392s	user 0.008s	sys 0.042s
I20260709 12:10:11.439549 10431 raft_consensus.cc:2376] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
W20260709 12:10:11.474565 11838 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.798s	user 0.015s	sys 0.045s
I20260709 12:10:11.474943 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1124) took 1212 ms. Trace:
I20260709 12:10:11.460582 10513 rpcz_store.cc:276] 0709 12:10:09.401727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.401898 (+   171us) service_pool.cc:224] Handling call
0709 12:10:10.805038 (+1403140us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400360454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:10.805212 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2739838720}
I20260709 12:10:11.475339 10431 rpcz_store.cc:276] 0709 12:10:10.262398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.421085 (+158687us) service_pool.cc:224] Handling call
0709 12:10:11.474868 (+1053783us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.475962 12011 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.476452 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:11.476614 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:10.823364 11910 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.477383 10431 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:11.477922 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1130) took 1213 ms. Trace:
I20260709 12:10:11.478159 11942 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:11.478238 10431 rpcz_store.cc:276] 0709 12:10:10.264257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.476337 (+1212080us) service_pool.cc:224] Handling call
0709 12:10:11.477864 (+  1527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.019507 11930 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:11.440248 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.455523 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1128) took 1191 ms. Trace:
I20260709 12:10:11.455996 12057 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:11.127333 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:10.973752 12034 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.949s	user 0.011s	sys 0.015s
W20260709 12:10:10.997206 10401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.027s	sys 0.032s
W20260709 12:10:11.427506 10553 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.987s (SENT)
W20260709 12:10:11.265300 11810 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.519s	user 0.015s	sys 0.005s
W20260709 12:10:11.123672 11950 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.772s	user 0.028s	sys 0.039s
I20260709 12:10:11.468559 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.460678 10437 rpcz_store.cc:276] 0709 12:10:10.157711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.186861 (+ 29150us) service_pool.cc:224] Handling call
0709 12:10:11.413420 (+1226559us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:11.468734 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 1.86973e+06us
I20260709 12:10:11.419471 11987 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:11.468658 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.490512 10507 raft_consensus.cc:2410] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:11.486141 10508 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:11.375425 11907 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.491360 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:11.167825 12024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.053s	sys 0.103s
W20260709 12:10:10.803814 11937 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.790s	user 0.010s	sys 0.042s
I20260709 12:10:10.998584 12004 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.457690 11984 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:11.458477 11827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.005s	sys 0.033s
I20260709 12:10:11.479615 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.479841 10436 rpcz_store.cc:276] 0709 12:10:10.263644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.657389 (+393745us) service_pool.cc:224] Handling call
0709 12:10:11.455428 (+798039us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.534512 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:11.457175 10510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.057s	user 0.012s	sys 0.054s
I20260709 12:10:11.535897 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1129) took 1272 ms. Trace:
W20260709 12:10:10.821250 10479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.296s	user 0.014s	sys 0.055s
I20260709 12:10:11.536381 10432 rpcz_store.cc:276] 0709 12:10:10.263939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.479344 (+1215405us) service_pool.cc:224] Handling call
0709 12:10:11.535818 (+ 56474us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:11.536816 10479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.012s	user 0.019s	sys 0.077s
I20260709 12:10:11.477404 10513 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:11.481637 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1407) took 1926 ms. Trace:
I20260709 12:10:11.537436 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:10.884869 11990 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.545s	user 0.013s	sys 0.039s
W20260709 12:10:11.484467 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:11.487135 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.544561 10506 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:11.537868 10430 rpcz_store.cc:276] 0709 12:10:09.556664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.556801 (+   137us) service_pool.cc:224] Handling call
0709 12:10:09.560761 (+  3960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:09.560999 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:10.408027 (+847028us) log.cc:844] Serialized 14 byte log entry
0709 12:10:11.481297 (+1073270us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:11.481352 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:11.481395 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.481463 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.481576 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33286,"prepare.queue_time_us":55643,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":13929,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":14379,"thread_start_us":1899874,"threads_started":2}
I20260709 12:10:11.545569 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.546350 10506 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:11.484644 10401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.442s	user 0.035s	sys 0.042s
I20260709 12:10:11.484838 10438 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:11.485083 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:11.479199 11910 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:11.486320 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.482999 12017 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.489539 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:11.492957 10507 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:11.550197 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:11.551093 10513 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:11.494557 11937 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.481s	user 0.024s	sys 0.061s
W20260709 12:10:11.486876 11950 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.135s	user 0.034s	sys 0.045s
W20260709 12:10:10.795282 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68027.3us
I20260709 12:10:11.494493 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.497792 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.498023 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.498651 11907 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.501643 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.476540 12011 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.504468 10511 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:11.478811 11942 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.413025 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1125) took 1150 ms. Trace:
I20260709 12:10:11.506462 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:11.619851 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 113531us
I20260709 12:10:11.620949 11987 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:11.621140 10404 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:10:11.621418 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1408) took 2063 ms. Trace:
I20260709 12:10:11.538244 10432 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:11.557760 11975 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:11.560875 11827 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:11.495647 12024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.503s	user 0.058s	sys 0.106s
I20260709 12:10:11.550565 10437 raft_consensus.cc:2368] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:11.420396 10434 rpcz_store.cc:276] 0709 12:10:10.148340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.151225 (+  2885us) service_pool.cc:224] Handling call
0709 12:10:11.419901 (+1268676us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.483193 12061 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:11.554481 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:11.557153 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.555881 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:11.559638 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 132170us
W20260709 12:10:11.502024 10433 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
W20260709 12:10:11.560182 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70745.5us
I20260709 12:10:11.560223 10508 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:11.486577 11912 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:11.558287 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.640775 10512 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:11.649453 10506 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:11.594674 11216 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:11.594909 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.540793 12057 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:11.531224 11982 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.531489 11906 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.596554 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:11.544067 11990 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.204s	user 0.020s	sys 0.046s
I20260709 12:10:11.562384 10431 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:11.614969 10504 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:11.493072 11810 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.623164 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1137) took 1356 ms. Trace:
I20260709 12:10:11.535919 10436 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:11.555199 11212 raft_consensus.cc:1217] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:10:11.616669 10505 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:11.666427 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1133) took 1401 ms. Trace:
I20260709 12:10:11.666824 10436 rpcz_store.cc:276] 0709 12:10:10.265116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.534098 (+1268982us) service_pool.cc:224] Handling call
0709 12:10:11.666323 (+132225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.629088 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1132) took 1364 ms. Trace:
I20260709 12:10:11.667877 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.639072 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.546790 12004 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:11.653506 10553 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.012s (SENT)
I20260709 12:10:11.647503 11218 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:11.653879 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:11.671679 10513 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:11.672590 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:11.617800 11937 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:11.655071 10506 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:11.562256 11822 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.629442 11779 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:10:11.551245 12053 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 12:10:11.660408 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1134) took 1395 ms. Trace:
I20260709 12:10:11.661497 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1131) took 1396 ms. Trace:
I20260709 12:10:11.662130 10432 rpcz_store.cc:276] 0709 12:10:10.266319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.537115 (+1270796us) service_pool.cc:224] Handling call
0709 12:10:11.623094 (+ 85979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.625574 11913 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:11.621835 10435 rpcz_store.cc:276] 0709 12:10:09.558497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.561839 (+  3342us) service_pool.cc:224] Handling call
0709 12:10:09.604907 (+ 43068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:11.463415 (+1858508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.621134 (+157719us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400436a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:11.621350 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625134848}
I20260709 12:10:11.667754 10437 rpcz_store.cc:276] 0709 12:10:10.264834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.486057 (+1221223us) service_pool.cc:224] Handling call
0709 12:10:11.629019 (+142962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.669003 10511 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:11.711061 10430 rpcz_store.cc:276] 0709 12:10:10.265444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.546058 (+1280614us) service_pool.cc:224] Handling call
0709 12:10:11.660336 (+114278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.672997 10508 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:11.673556 11178 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.906s (SENT)
I20260709 12:10:11.675602 11218 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:10:11.676904 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:11.658953 11728 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.877s	user 0.008s	sys 0.007s
I20260709 12:10:11.688393 10504 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:11.638442 11910 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:10:11.679441 11987 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.654321 12061 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:11.730409 10438 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 1398) took 2198 ms (client timeout 1757 ms). Trace:
W20260709 12:10:11.734942 10438 rpcz_store.cc:269] 0709 12:10:09.472321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.472506 (+   185us) service_pool.cc:224] Handling call
0709 12:10:11.670569 (+2198063us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54003534d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:11.670686 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4500402944}
I20260709 12:10:11.730926 11913 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:11.732087 12069 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:10:11.736896 11913 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:11.737713 12061 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:11.599640 11984 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.650614 10433 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:11.639318 11975 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.734404 11942 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:11.733629 11987 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.767498 11810 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:11.732463 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:11.709860 12066 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:11.737860 11822 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.768692 12066 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:11.769170 12066 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.769474 11987 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.770503 11822 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.737538 11218 pending_rounds.cc:85] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 4
I20260709 12:10:11.739543 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.771301 11218 pending_rounds.cc:107] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:10:11.732928 10431 rpcz_store.cc:276] 0709 12:10:10.264600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.480744 (+1216144us) service_pool.cc:224] Handling call
0709 12:10:11.661427 (+180683us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.772058 10434 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:11.771881 11218 raft_consensus.cc:2889] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:11.772663 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1135) took 1507 ms. Trace:
I20260709 12:10:11.773066 10434 rpcz_store.cc:276] 0709 12:10:10.265750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.666881 (+1401131us) service_pool.cc:224] Handling call
0709 12:10:11.772590 (+105709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896}
I20260709 12:10:11.662534 11906 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.773598 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.775260 11913 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.775950 11906 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:11.776530 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.777030 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1123) took 1515 ms. Trace:
I20260709 12:10:11.777876 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.623874 10429 rpcz_store.cc:276] 0709 12:10:10.262701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.518011 (+255310us) service_pool.cc:224] Handling call
0709 12:10:11.412947 (+894936us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:11.766018 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 112513us
I20260709 12:10:11.778647 10431 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:11.729271 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:11.756052 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:11.810019 10511 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.659076 11912 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:11.809444 11219 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:11.835388 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:11.776619 11984 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.687286 11937 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:11.777449 10433 rpcz_store.cc:276] 0709 12:10:10.262038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.264860 (+  2822us) service_pool.cc:224] Handling call
0709 12:10:11.776952 (+1512092us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:11.858734 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 185216us
I20260709 12:10:11.662304 11982 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.740082 11991 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.865518 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:11.732847 12038 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.767931 11942 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:11.815223 11910 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.759519 11907 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.684526 12017 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.806238 11728 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3.
I20260709 12:10:11.824222 11888 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.816201 11930 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:11.764684 12004 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:11.874289 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.875157 10436 raft_consensus.cc:2376] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2. Re-sending same reply.
I20260709 12:10:11.858846 12062 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:11.882314 11177 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.907s (SENT) [suppressed 42 similar messages]
I20260709 12:10:11.869261 11932 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:11.758272 12008 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:11.864745 12060 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:10:11.866473 11937 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.870373 11942 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.947572 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.950690 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1127) took 1687 ms. Trace:
I20260709 12:10:11.951071 10437 rpcz_store.cc:276] 0709 12:10:10.263363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.730655 (+1467292us) service_pool.cc:224] Handling call
0709 12:10:11.947372 (+216717us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:11.950620 (+  3248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20239744}
I20260709 12:10:11.948269 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:11.946293 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.953094 10432 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:11.943148 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 81) took 2627 ms. Trace:
I20260709 12:10:11.953683 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1147) took 1262 ms. Trace:
I20260709 12:10:11.954075 10432 rpcz_store.cc:276] 0709 12:10:10.691292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.730978 (+1039686us) service_pool.cc:224] Handling call
0709 12:10:11.946090 (+215112us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:11.953628 (+  7538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19069824}
I20260709 12:10:11.883647 11779 tablet_bootstrap.cc:492] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:10:11.955344 10430 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5.
I20260709 12:10:11.958544 10432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
W20260709 12:10:11.959690 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 77420us
I20260709 12:10:11.959828 10432 raft_consensus.cc:606] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:11.960844 10432 raft_consensus.cc:614] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:11.926831 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:11.960783 10431 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:11.955514 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:11.962244 10437 raft_consensus.cc:2410] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:11.937599 11785 ts_tablet_manager.cc:1434] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 2.988s	user 0.020s	sys 0.054s
W20260709 12:10:11.963042 11785 ts_tablet_manager.cc:1467] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Tablet startup took 3018ms
W20260709 12:10:11.952558 12065 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 12:10:11.874667 11910 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:11.953930 10585 rpcz_store.cc:276] 0709 12:10:09.311245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.312025 (+   780us) service_pool.cc:224] Handling call
0709 12:10:09.312465 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:09.312737 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:11.467507 (+2154770us) log.cc:844] Serialized 14 byte log entry
0709 12:10:11.935852 (+468345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:11.935946 (+    94us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:11.936014 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.936120 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.936323 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2143886,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"thread_start_us":2570434,"threads_started":2}
W20260709 12:10:11.971969 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:11.900537 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:11.997743 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.019245 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1140) took 1680 ms. Trace:
I20260709 12:10:12.020097 10432 rpcz_store.cc:276] 0709 12:10:10.339115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.017939 (+1678824us) service_pool.cc:224] Handling call
0709 12:10:12.018454 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.018716 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.018762 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:12.018839 (+    77us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:12.018886 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.018965 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.019156 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:12.023646 11906 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:11.886044 11888 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.024492 11906 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.026117 11888 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.026716 11888 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.958423 11932 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:11.964627 11912 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:11.878858 12017 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:11.971361 12067 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:10:12.036900 11906 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.039031 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1139) took 1772 ms. Trace:
I20260709 12:10:12.045149 10430 rpcz_store.cc:276] 0709 12:10:10.266945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.969583 (+1702638us) service_pool.cc:224] Handling call
0709 12:10:12.038361 (+ 68778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.038595 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.038634 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:12.038691 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:12.038734 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.038804 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.038937 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.972129 12011 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.031669 11983 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.047798 10431 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:12.034976 11779 ts_tablet_manager.cc:1403] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 3.848s	user 0.064s	sys 0.084s
I20260709 12:10:12.050163 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.050431 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1126) took 1787 ms. Trace:
W20260709 12:10:12.043017 11838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.005s	sys 0.005s
W20260709 12:10:12.053530 11838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.005s	sys 0.005s
I20260709 12:10:12.052958 10431 rpcz_store.cc:276] 0709 12:10:10.263035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.967970 (+1704935us) service_pool.cc:224] Handling call
0709 12:10:12.027592 (+ 59622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.050080 (+ 22488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.050358 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:12.040798 11932 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.039520 12068 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:12.060621 10553 leader_election.cc:336] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.086s (SENT)
I20260709 12:10:12.057366 10430 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:12.047199 12057 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.061700 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1143) took 1494 ms. Trace:
I20260709 12:10:12.063359 10430 rpcz_store.cc:276] 0709 12:10:10.567770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.046459 (+1478689us) service_pool.cc:224] Handling call
0709 12:10:12.056318 (+  9859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.061401 (+  5083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.061630 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:12.065311 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116596us
W20260709 12:10:12.066757 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:12.067121 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.009s	sys 0.008s
W20260709 12:10:12.068817 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.009s	sys 0.008s
I20260709 12:10:12.061142 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:12.062543 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.007s	sys 0.042s
W20260709 12:10:12.070569 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.007s	sys 0.042s
W20260709 12:10:12.070748 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 596884us
I20260709 12:10:12.072463 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.087383 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.089910 10430 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:12.095634 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.039395 11888 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.090584 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1146) took 1447 ms. Trace:
I20260709 12:10:12.098543 10430 rpcz_store.cc:276] 0709 12:10:10.647219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.064499 (+1417280us) service_pool.cc:224] Handling call
0709 12:10:12.089524 (+ 25025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.090353 (+   829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.090533 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:12.101194 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1149) took 1296 ms. Trace:
W20260709 12:10:12.095351 12050 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.418s	user 0.002s	sys 0.000s
I20260709 12:10:12.073467 12062 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.055131 11983 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.103377 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.115521 10507 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
I20260709 12:10:12.084336 11982 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:12.104687 10436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1136) took 1838 ms (client timeout 2233 ms). Trace:
I20260709 12:10:12.127163 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:12.127753 10436 rpcz_store.cc:269] 0709 12:10:10.266040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.667594 (+1401554us) service_pool.cc:224] Handling call
0709 12:10:12.104476 (+436882us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000866d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:12.104624 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474924800}
I20260709 12:10:12.097484 11827 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:12.099568 10433 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:12.103008 12004 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.068589 12057 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.105142 10432 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:12.130004 10478 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.130081 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1150) took 1233 ms. Trace:
I20260709 12:10:12.100350 11910 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.135857 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.102641 12011 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.138413 10436 rpcz_store.cc:276] 0709 12:10:10.896378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.128777 (+1232399us) service_pool.cc:224] Handling call
0709 12:10:12.129150 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.129564 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.129605 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:12.129672 (+    67us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:12.129720 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.129809 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.129994 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1093}
I20260709 12:10:12.101838 10430 rpcz_store.cc:276] 0709 12:10:10.821744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.099611 (+1277867us) service_pool.cc:224] Handling call
0709 12:10:12.100379 (+   768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.100796 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.100823 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:12.100872 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:12.100904 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.100974 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.101136 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96}
I20260709 12:10:12.095252 10522 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-0b474d9b241d4a698702287ee1bbe3b2 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:10:12.128422 12075 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.151960 10436 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:12.143862 10507 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:12.124248 12082 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.069290 12068 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:12.150931 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.163193 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.131642 12074 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.133157 11991 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.155323 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.174011 10585 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:12.153978 10430 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:12.177017 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.177683 10585 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:12.149119 12008 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.172812 11822 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.177424 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.224998 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.232481 10579 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:12.233189 12057 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.233543 11983 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.233705 11913 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:12.177040 11950 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.161962 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1144) took 1553 ms. Trace:
I20260709 12:10:12.170135 10522 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-0b474d9b241d4a698702287ee1bbe3b2 on tablet 0b474d9b241d4a698702287ee1bbe3b2 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:12.228642 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.142606 12004 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.169533 11937 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.226941 12017 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.162856 11907 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.265494 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:12.266320 10511 raft_consensus.cc:606] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:12.241890 12086 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.166186 12082 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.126891 12062 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:12.246124 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 185528us
W20260709 12:10:12.270514 10553 leader_election.cc:336] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.114s (SENT)
I20260709 12:10:12.271152 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.225096 12088 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:12.155053 12011 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.278482 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.280383 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:12.281467 10553 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.128s (SENT)
I20260709 12:10:12.179250 11990 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:10:12.243924 12008 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.287851 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:12.250247 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.290292 10509 raft_consensus.cc:2410] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:12.162448 12075 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.266814 10511 raft_consensus.cc:614] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:12.181782 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.266934 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:12.295913 11218 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:12.225265 12077 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 12:10:12.234797 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1173) took 2875 ms. Trace:
I20260709 12:10:12.268793 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1145) took 1641 ms. Trace:
W20260709 12:10:12.299415 12080 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:10:12.256225 11888 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.274184 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:12.276839 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:12.243101 12073 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:10:12.279516 10508 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:12.257858 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.317864 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:12.266080 11937 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:12.320806 10585 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:12.269304 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:12.327880 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 84815.5us
I20260709 12:10:12.298434 10510 rpcz_store.cc:276] 0709 12:10:09.352373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.352576 (+   203us) service_pool.cc:224] Handling call
0709 12:10:09.353063 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:09.353467 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:09.353507 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:09.358307 (+  4800us) log.cc:844] Serialized 26 byte log entry
0709 12:10:11.536349 (+2178042us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:11.536416 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.536525 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:12.227467 (+690942us) raft_consensus.cc:1685] finished
0709 12:10:12.227494 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.227671 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15367,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":11847,"thread_start_us":2135712,"threads_started":1,"wal-append.queue_time_us":2239749}
I20260709 12:10:12.338295 10587 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:12.340108 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.684s (SENT)
I20260709 12:10:12.249665 10433 rpcz_store.cc:276] 0709 12:10:10.608337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.058927 (+1450590us) service_pool.cc:224] Handling call
0709 12:10:12.083858 (+ 24931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.139689 (+ 55831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.156149 (+ 16460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:12.340813 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.345424 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:12.336668 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:12.336678 12079 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 12:10:12.336699 12081 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:10:12.300202 11910 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:12.379143 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50021.4us
I20260709 12:10:12.405571 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:12.407326 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:12.407945 12066 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.638s	user 0.016s	sys 0.000s
I20260709 12:10:12.408422 12066 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.410748 11910 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.411952 10584 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:12.413384 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.386281 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:12.397321 12089 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:10:12.402251 12060 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:12.403340 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.329659 11930 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.279698 12086 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.293679 12088 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:12.671645 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.672031 11220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 260) took 1129 ms. Trace:
I20260709 12:10:12.672175 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:12.416078 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:12.672431 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.675661 10584 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:12.677079 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.678220 10587 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:12.332458 11937 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.679301 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.680155 10586 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:12.683012 11779 raft_consensus.cc:359] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:12.683727 12066 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.684031 11779 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
W20260709 12:10:12.684753 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 344694us
W20260709 12:10:12.472002 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.672761 11220 rpcz_store.cc:276] 0709 12:10:11.542718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.550021 (+  7303us) service_pool.cc:224] Handling call
0709 12:10:11.639119 (+ 89098us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:11.683804 (+ 44685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:11.708677 (+ 24873us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:11.765319 (+ 56642us) log.cc:844] Serialized 26 byte log entry
0709 12:10:12.064060 (+298741us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:12.064132 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.064197 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:12.415377 (+351180us) raft_consensus.cc:1685] finished
0709 12:10:12.415401 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.671955 (+256554us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11927,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":22543,"thread_start_us":294399,"threads_started":1,"wal-append.queue_time_us":367543}
I20260709 12:10:12.703723 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:12.687462 12004 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:12.671772 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 390348us
I20260709 12:10:12.413789 11910 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.708250 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:12.582834 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:12.718173 11220 raft_consensus.cc:1217] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:10:12.449100 12090 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.706302 12068 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:12.717722 12093 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:10:12.716790 12100 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:12.705700 10582 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:12.726513 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:12.761525 10512 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:10:12.764472 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 346872us
W20260709 12:10:12.353120 12084 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:10:12.694312 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.750813 12100 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:12.415735 12085 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:10:12.695652 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:12.761185 11215 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:12.696519 11214 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:12.699994 11907 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:12.414227 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:11.969250 11785 ts_tablet_manager.cc:1469] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Trace:
0709 12:10:01.198885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:01.199051 (+   166us) service_pool.cc:224] Handling call
0709 12:10:01.322653 (+123602us) ts_tablet_manager.cc:934] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
0709 12:10:04.131570 (+2808917us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:04.550251 (+418681us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:04.550395 (+   144us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:04.795861 (+245466us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:04.796198 (+   337us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:04.796523 (+   325us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:04.796591 (+    68us) inbound_call.cc:177] Queueing success response
0709 12:10:05.421218 (+624627us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b180029caa0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f896ed8 00007f2c8f9f1de2 00007f2c97a56d23 00007f2c97a5655d 00007f2c97af23d8 00007f2c97b23f4e 00007f2c97b08fdd
0709 12:10:08.398438 (+2977220us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:10:08.949291 (+550853us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:10:09.078909 (+129618us) tablet_replica.cc:261] Starting instrumentation
0709 12:10:09.079641 (+   732us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":31442,"mutex_wait_us":27179,"raft.queue_time_us":11182,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6128,"spinlock_wait_cycles":12288256,"tablet-copy.queue_time_us":58880,"tablet-download-pool.queue_time_us":2446,"tablet-download-pool.run_cpu_time_us":2885,"tablet-download-pool.run_wall_time_us":2548251,"thread_start_us":115083,"threads_started":2}
I20260709 12:10:12.723237 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.558106 10404 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:12.725311 10436 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:12.681062 11937 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:12.752187 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 346754us
I20260709 12:10:12.718017 12075 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.331339 12091 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:12.400382 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:12.770344 10581 raft_consensus.cc:2410] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:12.772145 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:12.776774 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55031.4us
I20260709 12:10:12.692844 12086 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.813108 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:12.778623 10479 connection.cc:256] RPC call timeout handler was delayed by -1.2304891080000289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:10:12.814087 10479 connection.cc:256] RPC call timeout handler was delayed by -1.0585891079999783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 12:10:12.772549 12100 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:12.801506 10512 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:12.374413 10505 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
I20260709 12:10:12.758049 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:12.820262 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.821199 10584 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:12.780434 12082 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:12.783911 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 225940us
I20260709 12:10:12.716231 11779 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:12.790627 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.833721 10507 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:12.770812 12062 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.838888 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1138) took 2572 ms. Trace:
I20260709 12:10:12.772986 12011 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.808902 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.869382 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:12.810320 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.810490 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.871582 10587 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:12.872524 10580 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.783878 10510 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:12.823282 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:12.874485 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51305.6us
W20260709 12:10:12.831517 10553 connection.cc:256] RPC call timeout handler was delayed by -2.4949596030000976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.337s for a response
I20260709 12:10:12.876020 11822 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:12.876073 11975 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:12.876459 11822 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:12.876961 11822 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:12.788961 11930 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:12.878645 11930 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.821348 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:12.834966 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.876120 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:12.880525 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:12.881276 10433 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.883209 12074 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.883702 12074 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:12.798549 12068 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:12.883792 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:12.884320 12068 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:12.884774 12068 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:12.876487 10553 leader_election.cc:336] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.186s (SENT)
I20260709 12:10:12.886207 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.835645 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:12.889083 12088 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.890115 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:12.891427 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:12.892323 10434 raft_consensus.cc:2410] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:12.893036 10434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:12.893337 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:12.893644 10434 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:12.893800 11932 consensus_meta.cc:287] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.828s	user 0.009s	sys 0.003s
I20260709 12:10:12.894177 12077 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:12.894193 10513 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:12.896013 12047 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.217s	user 0.005s	sys 0.026s
I20260709 12:10:12.896068 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.897362 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.897626 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:12.897989 10513 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:12.898916 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.899325 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.298487 10431 rpcz_store.cc:276] 0709 12:10:10.627794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.057767 (+1429973us) service_pool.cc:224] Handling call
0709 12:10:12.059471 (+  1704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.059735 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:12.170420 (+110685us) log.cc:844] Serialized 14 byte log entry
0709 12:10:12.267684 (+ 97264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:12.267736 (+    52us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:12.267771 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.267831 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.268731 (+   900us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":182352,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":527,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":17817,"thread_start_us":188171,"threads_started":2}
I20260709 12:10:12.900540 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.900907 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:12.835588 12095 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:10:12.837374 10435 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:10:12.838085 12072 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.026s	user 0.010s	sys 0.009s
I20260709 12:10:12.844585 11779 ts_tablet_manager.cc:1434] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.790s	user 0.024s	sys 0.000s
W20260709 12:10:12.919317 11779 ts_tablet_manager.cc:1467] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Tablet startup took 866ms
I20260709 12:10:12.847098 12099 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:12.923470 10434 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.894246 11932 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.923734 12099 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.833142 12086 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.925946 12086 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.859972 10582 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:12.829708 12075 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:12.929574 12075 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.931231 10581 raft_consensus.cc:2410] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:12.931893 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.932140 12075 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:12.829473 12004 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:12.932942 12004 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.933962 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.798766 11907 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.878793 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:12.886950 10553 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.233s (SENT)
I20260709 12:10:12.935657 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.839355 10437 rpcz_store.cc:276] 0709 12:10:10.266628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.969756 (+1703128us) service_pool.cc:224] Handling call
0709 12:10:11.970115 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:11.970358 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:11.970400 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:11.971028 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 12:10:12.133365 (+162337us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:12.228359 (+ 94994us) log.cc:844] Serialized 14 byte log entry
0709 12:10:12.231286 (+  2927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.231413 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:12.838653 (+607240us) raft_consensus.cc:1685] finished
0709 12:10:12.838677 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.838824 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2628,"prepare.queue_time_us":156139,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":304,"raft.queue_time_us":115,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":2221,"thread_start_us":185195,"threads_started":2,"wal-append.queue_time_us":230200}
W20260709 12:10:12.936291 11984 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.074s	user 0.013s	sys 0.002s
I20260709 12:10:12.936435 10584 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:12.936628 11907 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:12.936838 10437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:12.937230 10438 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:12.937479 10437 raft_consensus.cc:606] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:12.937855 11888 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:12.936300 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49390.1us
I20260709 12:10:12.938545 11932 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:12.937866 12004 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:12.939182 10553 leader_election.cc:336] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.547s (SENT)
I20260709 12:10:12.940827 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.942425 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:12.942700 10553 connection.cc:256] RPC call timeout handler was delayed by -2.1188833389999218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 12:10:12.943631 10553 connection.cc:256] RPC call timeout handler was delayed by -1.7201833389999592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
W20260709 12:10:12.943990 10553 connection.cc:256] RPC call timeout handler was delayed by -1.5591621759999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 12:10:12.944447 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2401709919997757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2388s for a response
I20260709 12:10:12.945402 10553 leader_election.cc:400] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3.
I20260709 12:10:12.964490 10511 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:12.965212 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:12.792342 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:12.910863 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.966230 11942 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.966620 11984 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.967070 12099 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:12.967670 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 175414us
I20260709 12:10:12.911705 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:12.814424 12091 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.972505 11910 raft_consensus.cc:993] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:10:12.972999 11910 raft_consensus.cc:1081] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:10:12.975222 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:12.976838 12091 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:12.977403 11215 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:12.978785 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:12.975167 12068 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:12.979682 11906 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:12.980129 11906 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:12.980225 12057 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.980532 11906 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:12.912341 10435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1148) took 2168 ms (client timeout 2388 ms). Trace:
W20260709 12:10:12.981186 10435 rpcz_store.cc:269] 0709 12:10:10.744352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.729874 (+985522us) service_pool.cc:224] Handling call
0709 12:10:12.750319 (+1020445us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:12.912251 (+161932us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21663488}
I20260709 12:10:12.913211 11990 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:12.916507 11930 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:12.919663 11779 ts_tablet_manager.cc:1469] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Trace:
0709 12:10:00.355072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.361645 (+  6573us) service_pool.cc:224] Handling call
0709 12:10:01.278522 (+916877us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb1a2 00007f2c988fc889 00007f2c97a2dd88 00007f2c97af1a07 00007f2c97b23f4e 00007f2c97b08fdd 00007f2c97b08f89 00007f2c97b08f19 00007f2c97b08ee1
0709 12:10:01.304210 (+ 25688us) ts_tablet_manager.cc:934] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
0709 12:10:04.291184 (+2986974us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:04.804714 (+513530us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:04.804850 (+   136us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:05.364196 (+559346us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:05.364714 (+   518us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:05.404303 (+ 39589us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:05.416060 (+ 11757us) inbound_call.cc:177] Queueing success response
0709 12:10:08.173768 (+2757708us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:10:12.054955 (+3881187us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:10:12.230094 (+175139us) tablet_replica.cc:261] Starting instrumentation
0709 12:10:12.316583 (+ 86489us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":18998,"raft.queue_time_us":6047,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":163,"spinlock_wait_cycles":47220480,"tablet-copy.queue_time_us":503148,"tablet-download-pool.queue_time_us":277,"tablet-download-pool.run_cpu_time_us":3297,"tablet-download-pool.run_wall_time_us":2466073,"thread_start_us":295973,"threads_started":2}
I20260709 12:10:12.937954 10437 raft_consensus.cc:614] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:12.884141 12074 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.943176 12075 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.945924 12004 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.946331 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1142) took 2486 ms. Trace:
I20260709 12:10:12.903023 10436 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:12.938444 11888 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.951798 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.062227 10504 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:12.962085 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.888615 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:12.967103 11984 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:12.963622 12086 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:12.979461 12017 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:12.911613 12102 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:12.913594 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1187) took 1255 ms. Trace:
I20260709 12:10:13.079046 10506 rpcz_store.cc:276] 0709 12:10:11.658078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.912747 (+1254669us) service_pool.cc:224] Handling call
0709 12:10:12.913067 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.913272 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.913300 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:12.913344 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:12.913381 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.913439 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.913535 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:12.915162 11950 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:12.984997 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:13.005025 12078 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.843s	user 0.009s	sys 0.010s
I20260709 12:10:13.006731 12068 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:13.007392 11822 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:13.083402 10436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:13.058385 12099 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:10:13.084185 10436 raft_consensus.cc:606] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:13.084610 10436 raft_consensus.cc:614] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:13.085109 11822 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:13.009810 12096 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.233s	user 0.000s	sys 0.000s
I20260709 12:10:13.086448  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:13.010136 12095 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:13.087671 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 84) took 1020 ms. Trace:
I20260709 12:10:13.016914 10433 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:13.017333 11906 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:12.983260 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:12.995553 12100 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.024708 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:13.024739 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.043990 10511 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:13.045297 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.051985 12088 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:13.054100 12011 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:13.058039 10432 rpcz_store.cc:276] 0709 12:10:10.475772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.021143 (+1545371us) service_pool.cc:224] Handling call
0709 12:10:12.104681 (+ 83538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.137859 (+ 33178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.946104 (+808245us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54000866d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:12.946263 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191338240}
I20260709 12:10:13.063349 10507 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:13.063432 10512 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
W20260709 12:10:13.064589 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 162604us
I20260709 12:10:13.067008 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:13.092541 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 109410us
I20260709 12:10:12.986129 10583 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:13.088032 10430 rpcz_store.cc:276] 0709 12:10:12.066952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.176055 (+109103us) service_pool.cc:224] Handling call
0709 12:10:12.225425 (+ 49370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:12.225862 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.225900 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:12.226453 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 12:10:12.392239 (+165786us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:12.698347 (+306108us) log.cc:844] Serialized 14 byte log entry
0709 12:10:12.698920 (+   573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.699036 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:13.009202 (+310166us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:13.087360 (+ 78158us) raft_consensus.cc:1685] finished
0709 12:10:13.087467 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:13.087600 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":183413,"prepare.queue_time_us":305451,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":273,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":9529,"spinlock_wait_cycles":3244288,"thread_start_us":145313,"threads_started":1,"wal-append.queue_time_us":516467}
I20260709 12:10:13.070493 11984 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.039441 12008 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:10:13.058715 12090 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:10:13.115732 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.115842 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.112777 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:13.118072 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:13.110001 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.121601 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:13.136300 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 111663us
I20260709 12:10:13.137244 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.139093 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.140501 10585 raft_consensus.cc:2410] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:13.116724 10584 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:13.148177 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:13.148772 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.149423 10580 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.151085 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.154451 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.155293 10585 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:13.156278 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.157058 10585 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:13.104558 12102 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:13.104192 12095 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:13.069324 12004 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.165203 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:13.170658 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.172672 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:13.176363 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:13.140885 11950 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.181524 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.193769 12095 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.197489 12102 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.171712 11910 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:13.204061 11910 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:13.204555 11910 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.150434 12064 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:10:13.162388 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:13.210501 12100 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:13.211380 11991 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:13.211934 11906 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.155433 11990 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:13.213230 11990 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:13.213737 11990 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:13.219193 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.222601 12004 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:13.227679 11907 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:13.228318 11907 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:13.228829 11907 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.229144 11930 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.232617 10583 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.238960 10580 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:13.162472 11975 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:13.245863 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.254314 10583 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
W20260709 12:10:13.263832 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.257582 10587 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:13.257592 10584 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.266314 10587 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:10:13.259007 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 189250us
W20260709 12:10:13.257282 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 272346us
I20260709 12:10:13.269004 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:13.269662 11218 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:13.266319 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 147185us
I20260709 12:10:13.213754 11906 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:10:13.274135 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:13.259305 11975 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:13.281502 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 168802us
I20260709 12:10:13.283146 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:13.284140 12102 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:13.289866 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:13.271905 12017 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:13.295442 10552 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:13.299063 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:13.300542 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:13.306597 11221 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:13.306828 11214 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:13.308550 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.309499 10582 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:13.309958 11991 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:13.310570 11991 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:13.303342 12100 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.323721 11218 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.325353 10584 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:13.316992 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:13.300498 12017 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:13.301981 12064 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:10:13.445223  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:13.422066 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.486245 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.546653  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:13.457479 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.402383 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.558616 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.480086 10430 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:13.456853 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.480034 10431 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:10:13.518937 10436 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:13.470561 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:13.477130 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 203057us
I20260709 12:10:13.562656 10506 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
W20260709 12:10:13.506876 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62991.7us
I20260709 12:10:13.446206 10332 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:13.427772 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.385675 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 1415) took 1120 ms. Trace:
I20260709 12:10:13.566473 10438 rpcz_store.cc:276] 0709 12:10:12.265259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.281860 (+ 16601us) service_pool.cc:224] Handling call
0709 12:10:13.385456 (+1103596us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b5400436054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:13.385593 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411228416}
I20260709 12:10:13.432654 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.346591 10580 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:13.362325 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47996.7us
I20260709 12:10:13.461773 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:13.362637 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 71386.2us
I20260709 12:10:13.371212 10581 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:10:13.399495 12123 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 12:10:13.403267 12124 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:10:13.551262 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 230) took 6433 ms. Trace:
I20260709 12:10:13.338155 12108 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.315913 12120 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:13.355808 11975 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:13.417011 11990 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:13.588629 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 310657us
I20260709 12:10:13.597054 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.312500 12077 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:13.599725 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:13.386313 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.480454 10432 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:13.576053 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:13.553399 12017 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:13.597457 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.620169 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.572551 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.618918 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.622260 10504 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 1.
I20260709 12:10:13.604804 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:13.531662 12068 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:13.596352 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:13.600873 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.625479 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:13.575037 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:13.578547 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 261621us
I20260709 12:10:13.605185 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:13.565419 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.363251 12099 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:13.516515 12112 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.371s	user 0.004s	sys 0.000s
I20260709 12:10:13.566497 10332 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:13.517376 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:13.488826 12114 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:13.420801 11822 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:13.532112 11950 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:13.645193 10506 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:13.659246 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:13.645840 10332 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:13.643651 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.649603 11218 pending_rounds.cc:85] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 4
I20260709 12:10:13.661247 11218 pending_rounds.cc:107] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:10:13.398715 11888 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.465617 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.595697 10335 rpcz_store.cc:276] 0709 12:10:07.102960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.105914 (+  2954us) service_pool.cc:224] Handling call
0709 12:10:13.536363 (+6430449us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:07.907861 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:07.931626 (+ 23765us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:07.931677 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:10:07.931694 (+    17us) tablet.cc:662] Decoding operations
0709 12:10:08.056680 (+124986us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:08.060608 (+  3928us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:08.071525 (+ 10917us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:10:08.230416 (+158891us) tablet.cc:701] Row locks acquired
0709 12:10:08.233268 (+  2852us) write_op.cc:260] PREPARE: finished
0709 12:10:08.261978 (+ 28710us) write_op.cc:270] Start()
0709 12:10:08.279146 (+ 17168us) write_op.cc:276] Timestamp: P: 1783599008251406 usec, L: 0
0709 12:10:08.282249 (+  3103us) op_driver.cc:348] REPLICATION: starting
0709 12:10:08.381590 (+ 99341us) log.cc:844] Serialized 2823 byte log entry
0709 12:10:08.671065 (+289475us) op_driver.cc:464] REPLICATION: finished
0709 12:10:09.344369 (+673304us) write_op.cc:301] APPLY: starting
0709 12:10:09.363235 (+ 18866us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:09.422623 (+ 59388us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:09.422825 (+   202us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:09.569639 (+146814us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:09.764370 (+194731us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:10:09.771658 (+  7288us) write_op.cc:312] APPLY: finished
0709 12:10:09.941398 (+169740us) log.cc:844] Serialized 59 byte log entry
0709 12:10:11.657469 (+1716071us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:11.666947 (+  9478us) write_op.cc:454] Released schema lock
0709 12:10:11.671896 (+  4949us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:11.735975 (+ 64079us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:11.736016 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28814,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":613482,"apply.run_cpu_time_us":172733,"apply.run_wall_time_us":2421589,"num_ops":7,"prepare.queue_time_us":282373,"prepare.run_cpu_time_us":15875,"prepare.run_wall_time_us":549092,"raft.queue_time_us":75360,"raft.run_cpu_time_us":26020,"raft.run_wall_time_us":508776,"replication_time_us":377588,"thread_start_us":2268984,"threads_started":5,"wal-append.queue_time_us":1635264,"wal-append.run_cpu_time_us":33699,"wal-append.run_wall_time_us":2679737}]]}
I20260709 12:10:13.362926 12088 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.285s
I20260709 12:10:13.659952 10506 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.619748 12120 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:13.618906 11930 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:13.661530 11218 raft_consensus.cc:2889] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:13.517874 11991 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:13.681284 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55905.5us
I20260709 12:10:13.636515 12068 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:13.643034 12099 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:13.663506 10332 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:13.688482 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:13.643168 12117 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.694547 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.572530 11906 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 12:10:13.432770 11984 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.441478 11910 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.432497 12074 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:13.623535 12127 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:10:13.715549 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:13.655900 12114 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:13.621315 12082 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:10:13.659065 11950 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:13.659931 11822 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:13.719568 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 143627us
I20260709 12:10:13.573299 11907 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:13.741459 10509 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:10:13.792712 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104306us
I20260709 12:10:13.716768 10332 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 3, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:13.695252 11888 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:13.731732 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:13.730777 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57465.7us
W20260709 12:10:13.756551 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 151834us
W20260709 12:10:13.743559 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.686303 12120 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.710661 12099 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.799407 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:13.728519 12008 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:13.686501 11930 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:13.716851 12077 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:13.801326 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 69694.1us
I20260709 12:10:13.731561 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:13.623930 12108 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:13.795732 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:13.803175 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:13.806789 10332 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:13.793280 12121 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:13.798398 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:13.823493 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.831220 10513 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:13.810088 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.801983 11888 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:13.833618 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.850551 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.827852 10332 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 2 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:13.853740 10332 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 3, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 3 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:13.740748 12088 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:13.830760 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:13.825796 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.805512 12008 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:13.742471 12102 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:13.842517 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.809870 12091 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:13.883786 12091 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:13.883991 12133 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:10:13.884323 12091 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.884989 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.885838 10508 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:13.810376 11930 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.886510 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.888144 11907 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:13.720187 12117 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.889156 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.859967 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.892103 10581 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.741556 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:13.832113 11932 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:13.865367 12132 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:10:13.865952 12077 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.867386 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.856809 11906 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:13.868064 12142 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:13.871358 12082 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:13.875430 10587 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.942255 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.892634 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.944499 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.895203 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.949446 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.882778 10506 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:13.842692 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:13.951510 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 108891us
I20260709 12:10:13.847715 11888 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.955312 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:13.902792 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.957118 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:13.905256 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.958142 10578 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:13.905542 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.878111 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:13.959287 10509 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:13.950881 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.924885 11910 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.935377 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.929133 10586 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.885370 12108 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.955554 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:13.969692 10511 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:13.970614 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.960237 10581 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.971789 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.958711 10583 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:13.961203 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 165543us
I20260709 12:10:13.898264 12088 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:13.975215 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 177015us
I20260709 12:10:13.975879 10584 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:13.823024 12068 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:13.987514 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 130647us
I20260709 12:10:13.937541 11906 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:13.990304 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.988214 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:13.945662 12057 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:13.989888 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.991401 10587 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.997151 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.997854 10511 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.998749 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:14.002170 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.968528 11990 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.956693 11932 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:13.992153 12064 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:14.002749 10511 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:13.962543 12066 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:14.009999 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 157420us
I20260709 12:10:14.003681 11984 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:13.873896 11822 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:13.989431 12121 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.986586 12113 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:14.017609 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:13.993911 11906 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.009029 12064 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:14.010075 12066 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:14.013810 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:13.989107 12053 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:10:13.989286 12062 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:10:13.856467 12114 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:14.008674 12011 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:10:14.071660 10587 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
W20260709 12:10:14.056900 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 226215us
W20260709 12:10:14.042475 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68401.6us
W20260709 12:10:14.020165 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 382174us
W20260709 12:10:14.042675 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65837.3us
I20260709 12:10:14.085029 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:13.986882 12008 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.046751 10510 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:10:14.074812 10404 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:14.094966 10436 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:14.042397 10508 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:14.056452 12066 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:14.051412 12064 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.051484 12074 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:14.123169 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.110612 10510 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:14.126626 10510 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:14.125458 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 135076us
I20260709 12:10:14.018939 10509 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:14.106931 12102 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:14.118299 12008 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.044764 12075 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 12:10:14.111569 12119 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:14.055668 12088 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.124534 12062 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:14.127099 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:14.151708 12004 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:14.173086 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:14.158541 12114 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:14.124832 12108 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:10:14.180554 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:14.119480 12144 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:10:14.133445 12074 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 12:10:14.167016 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 233105us
I20260709 12:10:14.186333 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:14.193370 10437 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:10:14.183439 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55586.1us
I20260709 12:10:14.194809 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:14.196192 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.202792 10431 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:14.205216 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.166185 12062 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:14.171978 12004 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:14.245399 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50697us
I20260709 12:10:14.245703 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:14.243916 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56719us
I20260709 12:10:14.180186 12119 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:14.193397 12128 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:14.187326 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 96196us
I20260709 12:10:14.263768 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.264197 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.267119 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:14.269207 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.271760 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.242178 11975 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:14.273627 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:14.258973 11984 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.257851 12139 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:14.242297 12157 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 12:10:14.295511 12151 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:14.279598 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.276126 12141 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
W20260709 12:10:14.302976 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 171027us
I20260709 12:10:14.264272 12017 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:10:14.301097 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45972.1us
I20260709 12:10:14.277266 11910 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:14.272842 12117 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:14.276199 11932 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:14.256103 11907 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 12:10:14.299320 12140 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:10:14.258345 12121 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:10:14.281836 12142 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:14.328157 12135 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:10:14.329432 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:14.277570 12136 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:14.277740 11975 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:14.273406 12102 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 12:10:14.349071 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.299250 12139 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:14.403276 12139 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.257726 12137 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
W20260709 12:10:14.318763 12146 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:10:14.312932 12128 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:10:14.343518 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 69969us
I20260709 12:10:14.321209 11910 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:14.411242 12128 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:14.411643 11910 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.419818 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:14.421255 11932 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:14.423714 11932 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:14.425697 12151 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:14.426270 12151 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:14.427790 12151 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.430565 12151 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.345278 12017 raft_consensus.cc:2764] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:10:14.488849 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 159501us
I20260709 12:10:14.495996 10438 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:14.349438 12142 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:14.507613 11218 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:14.374564 11975 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.367609 12136 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:14.489950 12135 consensus_queue.cc:1243] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:14.301594 12062 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.275828 12119 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.511548 12142 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:14.309291 12004 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.458956 11910 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.557579 12165 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:10:14.531805 12136 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:14.577704 12062 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.577178 12004 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:14.640296 10510 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.513s	user 0.013s	sys 0.000s
I20260709 12:10:14.641011 10510 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.650198 12148 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:14.671908 10510 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:14.662631 12053 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:14.662300 12165 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:14.680009 12153 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
I20260709 12:10:14.679764 12053 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:14.680552 12164 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:10:14.682459 12053 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:14.683456 12167 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:10:14.684948 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.686961 12064 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:10:14.687340 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1217) took 1087 ms. Trace:
I20260709 12:10:14.688271 10504 rpcz_store.cc:276] 0709 12:10:13.599663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.622939 (+ 23276us) service_pool.cc:224] Handling call
0709 12:10:13.650624 (+ 27685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:14.059901 (+409277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:14.059937 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:14.076044 (+ 16107us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.259270 (+183226us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:14.259329 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.259433 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:14.687060 (+427627us) raft_consensus.cc:1685] finished
0709 12:10:14.687084 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.687264 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83808,"raft.queue_time_us":238291,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":24234,"thread_start_us":532162,"threads_started":2,"wal-append.queue_time_us":252272}
W20260709 12:10:14.691447 12163 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 12:10:14.691571 11910 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:14.695431 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1223) took 1056 ms. Trace:
I20260709 12:10:14.671684 11984 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:14.680018 12155 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
I20260709 12:10:14.695899 10511 rpcz_store.cc:276] 0709 12:10:13.639450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.017785 (+378335us) service_pool.cc:224] Handling call
0709 12:10:14.018126 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:14.018388 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:14.018424 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:14.072875 (+ 54451us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.255249 (+182374us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:14.255303 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.255416 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:14.695094 (+439678us) raft_consensus.cc:1685] finished
0709 12:10:14.695133 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.695343 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":133041,"threads_started":1,"wal-append.queue_time_us":216711}
I20260709 12:10:14.697729 12165 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:14.733156 11888 consensus_meta.cc:287] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.005s
I20260709 12:10:14.733862 12165 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:14.733764 11888 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.717767 12064 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.739776 12139 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.746014 12178 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.669589 12169 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:14.721917 12091 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.836s	user 0.014s	sys 0.001s
I20260709 12:10:14.746877 12178 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.768438 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:14.759742 12091 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:14.772210 11930 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.886s	user 0.011s	sys 0.003s
I20260709 12:10:14.772720 11930 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.773873 11221 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:14.774010 12091 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:14.776803 12099 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.954s	user 0.008s	sys 0.006s
I20260709 12:10:14.777390 12099 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.779246 11219 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:14.688297 12148 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.783838 12178 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:14.785205 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1219) took 1147 ms. Trace:
I20260709 12:10:14.785665 10512 rpcz_store.cc:276] 0709 12:10:13.638068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.730612 (+ 92544us) service_pool.cc:224] Handling call
0709 12:10:13.795953 (+ 65341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:13.901628 (+105675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:13.901665 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:13.930436 (+ 28771us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.099450 (+169014us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:14.099494 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.099575 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:14.784981 (+685406us) raft_consensus.cc:1685] finished
0709 12:10:14.785002 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.785137 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125918,"raft.queue_time_us":178813,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":18506,"thread_start_us":111247,"threads_started":1,"wal-append.queue_time_us":199698}
I20260709 12:10:14.785229 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:14.788149 10511 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:14.789512 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:14.756575 12064 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:14.795423 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 78297.1us
I20260709 12:10:14.796655 12176 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:14.797317 12176 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.807008 12165 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:14.809383 12165 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.810746 11950 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.816044 11950 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.816303 12165 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:14.817777 12008 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.004s
I20260709 12:10:14.817981 12064 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:14.818806 12064 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:14.820375 11906 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.776s	user 0.008s	sys 0.004s
I20260709 12:10:14.805536 11937 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:14.833909 12099 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:14.820926 11906 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.849043 12176 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:14.851523 12120 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.026s	user 0.010s	sys 0.007s
I20260709 12:10:14.840734 11888 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:14.855327 12183 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:14.871943 11822 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.879833 12064 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:14.882762 11930 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:14.885468 11906 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:14.886721 12148 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:14.870074 12183 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:14.890197 12183 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:14.893707 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:14.894348 10511 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.856161 11937 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.861641 12120 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:10:14.904155 12114 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.003s
W20260709 12:10:14.912695 12066 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.790s	user 0.015s	sys 0.000s
I20260709 12:10:14.892995 11975 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.916850 10511 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:14.920771 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:14.924057 12120 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:14.920069 12066 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.925836 11937 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:14.927536 12139 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:14.912236 12142 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.931687 11975 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:14.933702 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.934311 12193 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:14.930826 12066 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:14.941327 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.941602 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.942832 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:14.938481 11212 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:14.939294 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:14.953203 12193 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:14.947634 12004 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:14.954706 12193 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:14.950569 12136 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:14.978086 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:14.965415 12004 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.009884 12192 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:15.006664 12136 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.025344 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.026188 10578 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:15.027397 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.030190 10584 raft_consensus.cc:2410] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:15.029048 10580 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:15.029542 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.032196 10585 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.024873 12086 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:15.042718 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.026690 11822 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.036942 12004 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:15.049321 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.054023 10582 raft_consensus.cc:2368] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:15.050697 11822 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.050403 12086 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:15.066591 12086 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:15.069792 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:15.073160 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.079089 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.079977 11907 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.080106 11822 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.080911 11907 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:15.081521 11907 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.083863 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.084697 10583 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:15.080139 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:15.091804 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.093108 10582 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:15.093017 11907 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:15.095413 12064 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:15.096911 11907 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:15.094013 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.000s	sys 0.013s
I20260709 12:10:15.098342 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:15.101076 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.000s	sys 0.015s
I20260709 12:10:15.100919 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:15.101621 10506 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:15.101718 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 830020us
I20260709 12:10:15.102473 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:15.100708 10478 leader_election.cc:400] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:15.104305 10478 leader_election.cc:403] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:15.104594 12100 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:15.104674 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:15.105073 12183 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.105968 10504 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:15.106675 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:15.107396 10506 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.108214 10583 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:15.109668 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.109112 12143 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:10:15.109468 12136 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:15.111779 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:15.113276 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.113731 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:15.114392 10504 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.115288 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:15.118172 12100 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:15.119293 12120 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:15.120151 12120 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:15.119776 12100 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.121595 12120 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.127120 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:15.134558 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:15.136826 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:15.142195 11975 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.142899 11975 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:15.143545 11975 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.140240 12136 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.134670 12114 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:15.155422 10582 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.146786 12136 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:15.156234 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:15.140084 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:15.139616 10587 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:15.159449 10580 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:15.159488 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:15.161491 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.164479 10584 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:15.085865 10585 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:15.169561 10506 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:15.167698 12114 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.170138 10504 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:15.156430 12136 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.166868 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:15.174391 11888 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.174520 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.174754 10403 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:15.174965 11888 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:15.175360 10403 leader_election.cc:403] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:15.175720 11888 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.176242 12119 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.180056 12121 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:15.180265 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:15.180919 12121 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:15.181479 12102 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:15.183966 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:15.182746 12121 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.185873 12143 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:15.192759 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:15.193183 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.206353 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.207186 10583 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.207083 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.218297 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:15.219249 10504 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:15.170735 11906 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.173270 11907 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:15.173146 12114 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:15.225582 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:15.231048 11178 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.623s (SENT)
I20260709 12:10:15.231740 11178 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:15.258279 10504 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:15.259922 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:15.236722 12161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.018s	sys 0.010s
I20260709 12:10:15.223439 11906 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:15.223690 12183 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:15.210558 12102 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:10:15.215662 12064 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:10:15.240831 12191 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:15.222798 12143 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:15.225339 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:15.237807 12077 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:15.225610 12114 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:15.264350 12161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.019s	sys 0.010s
I20260709 12:10:15.261235 12196 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:15.275259 10583 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:15.276340 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:15.278894 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:15.286209 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.285952 11888 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:15.267277 11906 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.273203 12102 consensus_queue.cc:1243] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:15.284667 10433 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.315402 10433 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:15.293013 11888 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.330281 11888 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.291530 12077 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:15.355298 12203 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:10:15.355672 12077 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.320859 11950 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:15.295343 12196 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:15.360263 11950 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:15.381026 11178 leader_election.cc:336] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.142s (SENT)
I20260709 12:10:15.381765 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:15.330865 11888 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:15.398329 12161 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:15.441828 12119 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:15.402706 12064 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.456836 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:15.457374 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 124383us
I20260709 12:10:15.444819 12202 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.463924 11950 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:15.402913 12143 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:15.471205 12206 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:10:15.470736 11888 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:15.481606 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 100643us
W20260709 12:10:15.501611 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:15.505074 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.508340 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.462563 12064 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.512104 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.504928 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.491133 12139 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.478721 12202 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.515224 11822 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:15.538333 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 148333us
I20260709 12:10:15.541007 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.464951 12196 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.518142 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:15.530493 12136 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.547566 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:15.548568 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.523689 12139 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.543020 11822 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.551790 12183 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:15.561235 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.562009 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.562502 12220 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.563221 12220 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.565608 12220 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.566159 12220 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.566644 12220 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:15.567510 12226 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:15.570928 10552 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.571441 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:15.577826 11950 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.578521 11950 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.581772 11950 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:15.587038 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:15.591470 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:15.592226 10505 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.561082 11888 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.561376 12119 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.596055 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.575541 10433 raft_consensus.cc:2368] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 4.
I20260709 12:10:15.574072 12082 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:15.572248 12127 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.572734 12183 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:15.631299 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.595299 12091 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:15.595523 12066 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:15.599071 11888 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:15.636793 12215 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:10:15.605787 12082 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:15.663992 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.666313 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.641808 12202 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.676225 12066 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:15.676607 12202 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:15.677790 11888 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.677484 12202 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.677608 12066 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.678615 11888 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:15.677634 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.684482 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.688476 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:15.684757 12211 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:10:15.685705 12183 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.606837 12119 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.668164 12082 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.698400 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:15.702877 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:15.710367 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.425s	user 0.015s	sys 0.047s
I20260709 12:10:15.658697 12091 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:15.688640 12064 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.731946 12121 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:15.753773 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.606035 12226 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.640499 12127 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:15.759678 11822 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.760977 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:15.764586 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.785178 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:15.796356 12139 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.753309 12136 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:15.869782 10582 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.713s	user 0.005s	sys 0.010s
I20260709 12:10:15.898954 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:15.904294 12060 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:15.869031 12114 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.007s
I20260709 12:10:16.004977 12119 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.059489 12114 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:16.060881 10553 connection.cc:256] RPC call timeout handler was delayed by -1.1327847600000496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2323s for a response
W20260709 12:10:16.061472 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0949768369998765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2061s for a response
W20260709 12:10:16.027397 11975 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.883s	user 0.012s	sys 0.005s
W20260709 12:10:16.037608 11906 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.737s	user 0.005s	sys 0.010s
I20260709 12:10:16.028951 12202 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.049536 11932 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:16.062927 12191 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:16.085233 12127 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:16.058418 12060 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.086277 11932 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.079942 12226 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.829192 12121 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.076478 11975 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.116534 12114 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.763651 12220 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:16.073278 12099 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:16.086513 12142 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:15.898138 12128 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:15.928704 12091 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.111320 12191 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.121913 12226 raft_consensus.cc:1064] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:16.124926 12082 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.121693 12220 raft_consensus.cc:1064] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:16.131593 12128 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:16.126358 12226 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.185683 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:16.188599 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.190217 10431 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:16.192622 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.907s	user 0.017s	sys 0.048s
I20260709 12:10:16.193730 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.194126  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:16.195061 10431 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:15.901373 10507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1212) took 2445 ms (client timeout 2366 ms). Trace:
I20260709 12:10:16.078998 11906 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.210700 10583 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:16.227072 12057 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:16.118979 12099 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.258019 12057 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:16.241895 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:16.257709 10507 rpcz_store.cc:269] 0709 12:10:13.456268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.456435 (+   167us) service_pool.cc:224] Handling call
0709 12:10:15.900953 (+2444518us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54002c6ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:15.901199 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920596864}
I20260709 12:10:16.294467 12178 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:16.296769 12057 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.296351 12178 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.297179 11932 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:16.298106 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.298837 10582 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:16.299773 12178 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.311375 12121 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.311960 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:16.313575 10584 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:16.315109 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.316442 10432 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:16.317133 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1164) took 1030 ms. Trace:
I20260709 12:10:16.317620 10432 rpcz_store.cc:276] 0709 12:10:15.286801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.214326 (+927525us) service_pool.cc:224] Handling call
0709 12:10:16.317046 (+102720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.318146 10431 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:16.318898 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1166) took 1031 ms. Trace:
I20260709 12:10:16.319347 10431 rpcz_store.cc:276] 0709 12:10:15.287615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.297583 (+1009968us) service_pool.cc:224] Handling call
0709 12:10:16.318794 (+ 21211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.293550 12060 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:16.300004 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1419) took 1145 ms. Trace:
I20260709 12:10:16.321494 10582 rpcz_store.cc:276] 0709 12:10:15.154314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.154539 (+   225us) service_pool.cc:224] Handling call
0709 12:10:15.155202 (+   663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.299491 (+1144289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.299876 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.323359 11975 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.324450 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:16.346578 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:16.349162 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:16.350566 10584 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:16.352015 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:16.353943 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:16.376037 10582 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:16.376631 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:16.379237 12082 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:16.379465 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:16.381215 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:16.381601 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:16.383279 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:16.384891 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:16.385910 10507 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:16.386940 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:16.387786 10507 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.314868 12099 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:16.407159 12191 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.407945 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 96) took 1206 ms. Trace:
I20260709 12:10:16.382910 10587 raft_consensus.cc:2410] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:16.422348 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.424165 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 93) took 1223 ms. Trace:
I20260709 12:10:16.378422 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:16.426223 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.426470 10507 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:16.434244 10430 rpcz_store.cc:276] 0709 12:10:15.200317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.422184 (+1221867us) service_pool.cc:224] Handling call
0709 12:10:16.422831 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.423365 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.423406 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.423486 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:16.423573 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.423664 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.423861 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.438510 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1154) took 1235 ms. Trace:
I20260709 12:10:16.439263 10432 rpcz_store.cc:276] 0709 12:10:15.202937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.421795 (+1218858us) service_pool.cc:224] Handling call
0709 12:10:16.438432 (+ 16637us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.458716 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:16.458633 10437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 90) took 1259 ms (client timeout 1623 ms). Trace:
W20260709 12:10:16.463109 10477 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.034s	sys 0.090s
W20260709 12:10:16.467175 10437 rpcz_store.cc:269] 0709 12:10:15.199274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.425493 (+1226219us) service_pool.cc:224] Handling call
0709 12:10:16.458484 (+ 32991us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:16.471212 10477 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.035s	sys 0.090s
W20260709 12:10:16.472301 10513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.013s	sys 0.066s
I20260709 12:10:16.472642 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:16.473469 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.032s	sys 0.044s
W20260709 12:10:16.473899 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.032s	sys 0.044s
W20260709 12:10:16.475566 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.477351 12099 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:16.473048 10513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.014s	sys 0.066s
I20260709 12:10:16.408601 10429 rpcz_store.cc:276] 0709 12:10:15.201322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.203962 (+  2640us) service_pool.cc:224] Handling call
0709 12:10:16.406156 (+1202194us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:16.406726 (+   570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.407223 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.407261 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.407344 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:16.407393 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.407500 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.407687 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21586816}
I20260709 12:10:16.481755 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:16.482908 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:16.473902 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1167) took 1186 ms. Trace:
I20260709 12:10:16.488456 10437 rpcz_store.cc:276] 0709 12:10:15.288007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.471674 (+1183667us) service_pool.cc:224] Handling call
0709 12:10:16.473826 (+  2152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:16.475826 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 904419us
I20260709 12:10:16.475886 10448 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-81b70ca03fbe4622a03717efd079d598 received from {username='slave'} at 127.0.0.1:54450
I20260709 12:10:16.490855 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 97) took 1289 ms. Trace:
I20260709 12:10:16.492110 10448 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-81b70ca03fbe4622a03717efd079d598 on tablet 81b70ca03fbe4622a03717efd079d598 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:16.521533 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.522894 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:16.492146 10429 rpcz_store.cc:276] 0709 12:10:15.201673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.482385 (+1280712us) service_pool.cc:224] Handling call
0709 12:10:16.490745 (+  8360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.526938 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1174) took 1234 ms. Trace:
I20260709 12:10:16.527550 10437 rpcz_store.cc:276] 0709 12:10:15.292232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.493424 (+1201192us) service_pool.cc:224] Handling call
0709 12:10:16.526846 (+ 33422us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:16.544549 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.000s	sys 0.023s
I20260709 12:10:16.544821 10448 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.EndTabletCopySession from 127.0.0.1:54450 (request call id 88) took 1346 ms. Trace:
W20260709 12:10:16.545153 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.000s	sys 0.023s
I20260709 12:10:16.545367 10448 rpcz_store.cc:276] 0709 12:10:15.198551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.475308 (+1276757us) service_pool.cc:224] Handling call
0709 12:10:16.544661 (+ 69353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.548321 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:16.540882 12077 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.006s	sys 0.018s
I20260709 12:10:16.539216 12121 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:16.536728 12099 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:16.561596 12077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.006s	sys 0.019s
I20260709 12:10:16.572863 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:16.572049 11178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.016s	sys 0.012s
I20260709 12:10:16.546870 12228 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:16.579181 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:16.579612 11178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.017s	sys 0.013s
W20260709 12:10:16.583756 12222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.002s	sys 0.021s
W20260709 12:10:16.584371 12222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.003s	sys 0.022s
I20260709 12:10:16.584227 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:16.585878 12227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.018s	sys 0.028s
W20260709 12:10:16.586359 12227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.018s	sys 0.028s
I20260709 12:10:16.586630 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:16.587903 12143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.002s	sys 0.020s
W20260709 12:10:16.588413 12143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.002s	sys 0.020s
I20260709 12:10:16.588922 11975 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:16.590492 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:16.583811 10479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.012s	sys 0.012s
W20260709 12:10:16.591565 10479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.012s	sys 0.012s
W20260709 12:10:16.591730 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.593276 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:16.593789 11975 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:16.594075 12148 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.320s	user 0.016s	sys 0.014s
W20260709 12:10:16.594568 12148 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.370s	user 0.017s	sys 0.014s
W20260709 12:10:16.583750 10578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.004s	sys 0.020s
I20260709 12:10:16.593115 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:16.595206 10578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.005s	sys 0.020s
I20260709 12:10:16.595722 11975 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:16.596442 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1410) took 1566 ms. Trace:
I20260709 12:10:16.596911 10578 rpcz_store.cc:276] 0709 12:10:15.030268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.030395 (+   127us) service_pool.cc:224] Handling call
0709 12:10:15.030719 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:15.030953 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:15.036604 (+  5651us) log.cc:844] Serialized 14 byte log entry
0709 12:10:15.714373 (+677769us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b1800333340. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f23a 00007f2c92ae9a2a
0709 12:10:16.595874 (+881501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.595983 (+   109us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:16.596095 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.596209 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.596383 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5383,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"spinlock_wait_cycles":14761856,"thread_start_us":881073,"threads_started":1}
I20260709 12:10:16.599077 11218 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:16.599356 10403 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:16.560467 11214 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:16.600979 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:16.601636 11214 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 272) took 2513 ms (client timeout 1845 ms). Trace:
I20260709 12:10:16.602100 11975 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:16.602197 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:16.608116 11218 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:16.611701 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.614121 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1430) took 1068 ms. Trace:
I20260709 12:10:16.591861 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 94) took 1391 ms. Trace:
W20260709 12:10:16.602175 11214 rpcz_store.cc:269] 0709 12:10:14.088085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.092277 (+  4192us) service_pool.cc:224] Handling call
0709 12:10:15.898628 (+1806351us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:16.601515 (+702887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15798656}
I20260709 12:10:16.615175 10587 rpcz_store.cc:276] 0709 12:10:15.542172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.452806 (+910634us) service_pool.cc:224] Handling call
0709 12:10:16.530154 (+ 77348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.609703 (+ 79549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.609752 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.609826 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:16.609873 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.609964 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.610130 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.625644 10507 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.624217 10437 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:16.627065 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1187) took 1254 ms. Trace:
I20260709 12:10:16.630219 10437 rpcz_store.cc:276] 0709 12:10:15.372444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.538353 (+1165909us) service_pool.cc:224] Handling call
0709 12:10:16.626956 (+ 88603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.637280 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:16.642164 11218 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:10:16.646667 11176 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.006s	sys 0.015s
I20260709 12:10:16.635941 12227 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:16.655483 11176 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.006s	sys 0.015s
I20260709 12:10:16.623307 12121 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.622838 10435 rpcz_store.cc:276] 0709 12:10:15.200641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.203819 (+  3178us) service_pool.cc:224] Handling call
0709 12:10:16.560289 (+1356470us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:16.591745 (+ 31456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21099392}
W20260709 12:10:16.680820 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:16.659340 12120 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.007s	sys 0.024s
I20260709 12:10:16.712921 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.716284 12226 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:16.716495 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:16.717572 10435 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:16.717415 12226 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.720562 12226 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:16.721210 12226 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:16.722023 12226 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.724578 12226 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:16.726249 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:16.730337 12066 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.023s	sys 0.047s
W20260709 12:10:16.730829 12066 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.023s	sys 0.047s
I20260709 12:10:16.734524 12227 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:16.740753 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 139984us
I20260709 12:10:16.742039 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:16.749202 10435 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:16.748971 11216 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:16.752424 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1189) took 1312 ms. Trace:
I20260709 12:10:16.753855 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 1785) took 1248 ms. Trace:
I20260709 12:10:16.761368 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:16.762109 10430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1179) took 1465 ms (client timeout 1945 ms). Trace:
I20260709 12:10:16.654935 12228 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.754421 10583 rpcz_store.cc:276] 0709 12:10:15.505216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.704399 (+199183us) service_pool.cc:224] Handling call
0709 12:10:16.753533 (+1049134us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400178754. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:16.753754 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407457664}
W20260709 12:10:16.762521 10430 rpcz_store.cc:269] 0709 12:10:15.296833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.463887 (+1167054us) service_pool.cc:224] Handling call
0709 12:10:16.762038 (+298151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.753916 10435 rpcz_store.cc:276] 0709 12:10:15.439832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.712848 (+1273016us) service_pool.cc:224] Handling call
0709 12:10:16.752050 (+ 39202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:16.755316 12120 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.168s	user 0.008s	sys 0.025s
I20260709 12:10:16.766795 10437 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:16.767246 12147 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:16.775449 12147 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.779681 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1192) took 1338 ms. Trace:
I20260709 12:10:16.780452 10437 rpcz_store.cc:276] 0709 12:10:15.441120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.634754 (+1193634us) service_pool.cc:224] Handling call
0709 12:10:16.779437 (+144683us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:16.779120 12147 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:16.716526 12243 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:16.787895 12243 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:16.821589 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58159.2us
W20260709 12:10:16.826152 10505 consensus_meta.cc:287] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.120s	user 0.015s	sys 0.009s
I20260709 12:10:16.826529 11991 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:16.827056 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 85296.7us
I20260709 12:10:16.827507 11991 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:16.826877 10505 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:16.831970 11888 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:16.833796 12226 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:16.834230 10433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1153) took 1631 ms (client timeout 1593 ms). Trace:
I20260709 12:10:16.834640 12226 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:16.820043 12227 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:16.846352 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:16.879385 11218 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:16.877833 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1433) took 2723 ms. Trace:
I20260709 12:10:16.882722 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:17.009716 12128 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.007s
W20260709 12:10:16.978240 11178 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.045s (SENT)
W20260709 12:10:16.915577 12196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.008s	sys 0.032s
I20260709 12:10:16.882594 12235 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.336491 12226 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:16.849088 12099 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:17.117460 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 257) took 2744 ms. Trace:
I20260709 12:10:17.375174 12236 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:16.871790 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:17.379724 10430 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:17.380357 10430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1191) took 1939 ms (client timeout 2012 ms). Trace:
W20260709 12:10:17.380754 10430 rpcz_store.cc:269] 0709 12:10:15.440853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.815312 (+1374459us) service_pool.cc:224] Handling call
0709 12:10:17.380255 (+564943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.382066 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:17.324436 12213 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:17.383205 10430 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:17.383749 10430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1163) took 2097 ms (client timeout 2272 ms). Trace:
W20260709 12:10:17.384298 10430 rpcz_store.cc:269] 0709 12:10:15.286393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.381619 (+2095226us) service_pool.cc:224] Handling call
0709 12:10:17.383665 (+  2046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.383828 12213 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:17.385478 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.391331 10430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1159) took 2119 ms (client timeout 2323 ms). Trace:
W20260709 12:10:17.377044 12196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.010s	sys 0.044s
W20260709 12:10:17.392230 10430 rpcz_store.cc:269] 0709 12:10:15.266693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.385117 (+2118424us) service_pool.cc:224] Handling call
0709 12:10:17.386299 (+  1182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.395449 12074 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.396242 12236 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:17.396402 12074 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:17.362430 10436 rpcz_store.cc:276] 0709 12:10:14.114922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.115191 (+   269us) service_pool.cc:224] Handling call
0709 12:10:14.115560 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:14.152856 (+ 37296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:14.152911 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:14.163327 (+ 10416us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.495310 (+331983us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:14.495361 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.495443 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:16.837435 (+2341992us) raft_consensus.cc:1685] finished
0709 12:10:16.837479 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.837723 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35546,"raft.queue_time_us":34066,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":8228,"thread_start_us":251467,"threads_started":1,"wal-append.queue_time_us":474581,"wal-append.run_cpu_time_us":1732,"wal-append.run_wall_time_us":1020406}
W20260709 12:10:16.834868 10433 rpcz_store.cc:269] 0709 12:10:15.202635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.458141 (+255506us) service_pool.cc:224] Handling call
0709 12:10:16.833704 (+1375563us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b540015f9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:16.833987 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23668992}
I20260709 12:10:17.399297 11216 raft_consensus.cc:2410] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:10:17.402395 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.405165 12088 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.095s	user 0.008s	sys 0.025s
W20260709 12:10:17.405649 12088 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.099s	user 0.008s	sys 0.025s
I20260709 12:10:17.405632 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.407307 10433 raft_consensus.cc:2410] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:17.407900 10433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1175) took 2115 ms (client timeout 2474 ms). Trace:
W20260709 12:10:17.408398 10433 rpcz_store.cc:269] 0709 12:10:15.292521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.405139 (+2112618us) service_pool.cc:224] Handling call
0709 12:10:17.407811 (+  2672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.409763 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.401386 12235 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:17.413589 12235 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:17.414602 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:17.403534 10436 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:17.415858 10436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1178) took 2122 ms (client timeout 2388 ms). Trace:
W20260709 12:10:17.416312 10436 rpcz_store.cc:269] 0709 12:10:15.296447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.398003 (+2101556us) service_pool.cc:224] Handling call
0709 12:10:17.415778 (+ 17775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.415499 10583 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:17.418749 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:17.419785 10583 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:17.426916 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.410863 10433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1161) took 2125 ms (client timeout 2621 ms). Trace:
W20260709 12:10:17.428776 10433 rpcz_store.cc:269] 0709 12:10:15.285426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.409311 (+2123885us) service_pool.cc:224] Handling call
0709 12:10:17.410787 (+  1476us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.431531 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:17.432752 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:17.431891 12213 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:17.437083 10430 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:17.436904 12236 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:17.437368 10434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 95) took 2236 ms (client timeout 1659 ms). Trace:
W20260709 12:10:17.437562 10430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1172) took 2145 ms (client timeout 2366 ms). Trace:
I20260709 12:10:17.438349 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.438854 10430 rpcz_store.cc:269] 0709 12:10:15.291721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.395205 (+2103484us) service_pool.cc:224] Handling call
0709 12:10:17.437492 (+ 42287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.439949 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:17.444257 10429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1177) took 2151 ms (client timeout 2061 ms). Trace:
I20260709 12:10:17.444703 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:17.444778 10429 rpcz_store.cc:269] 0709 12:10:15.293013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.443967 (+2150954us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:17.444158 (+   191us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:17.446090 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:17.446271 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:17.447114 10583 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:17.451339 12265 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.452083 12265 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:17.440325 10433 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:17.440675 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.438167 10434 rpcz_store.cc:269] 0709 12:10:15.200970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.436949 (+2235979us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:17.437134 (+   185us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:17.454239 10430 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:17.454439 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.454756 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1170) took 2165 ms. Trace:
I20260709 12:10:17.436055 10436 raft_consensus.cc:2376] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
I20260709 12:10:17.455231 10430 rpcz_store.cc:276] 0709 12:10:15.289271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.440235 (+2150964us) service_pool.cc:224] Handling call
0709 12:10:17.454683 (+ 14448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.455250 10434 raft_consensus.cc:2376] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
W20260709 12:10:17.455629 10436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1176) took 2162 ms (client timeout 2799 ms). Trace:
I20260709 12:10:17.457055 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.457123 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1158) took 2205 ms. Trace:
I20260709 12:10:17.458318 10430 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:17.458364 10434 rpcz_store.cc:276] 0709 12:10:15.251739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.454051 (+2202312us) service_pool.cc:224] Handling call
0709 12:10:17.457045 (+  2994us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.459144 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1169) took 2170 ms. Trace:
I20260709 12:10:17.459611 10430 rpcz_store.cc:276] 0709 12:10:15.288847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.456603 (+2167756us) service_pool.cc:224] Handling call
0709 12:10:17.459067 (+  2464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.460039 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.461051 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1180) took 2164 ms. Trace:
I20260709 12:10:17.461726 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.461946 12268 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:17.463093 10430 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:17.463227 10433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1171) took 2171 ms (client timeout 2856 ms). Trace:
I20260709 12:10:17.463629 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1168) took 2175 ms. Trace:
I20260709 12:10:17.464174 10430 rpcz_store.cc:276] 0709 12:10:15.288431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.461262 (+2172831us) service_pool.cc:224] Handling call
0709 12:10:17.463571 (+  2309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.439386 12213 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:17.457312 10436 rpcz_store.cc:269] 0709 12:10:15.292764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.427200 (+2134436us) service_pool.cc:224] Handling call
0709 12:10:17.455551 (+ 28351us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:17.464426 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 634887us
W20260709 12:10:17.463795 10433 rpcz_store.cc:269] 0709 12:10:15.291437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.430042 (+2138605us) service_pool.cc:224] Handling call
0709 12:10:17.463157 (+ 33115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.466434 12213 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:17.468365 10433 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:17.469115 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1156) took 2251 ms. Trace:
I20260709 12:10:17.469882 12213 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:17.469513 10433 rpcz_store.cc:276] 0709 12:10:15.217697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.467510 (+2249813us) service_pool.cc:224] Handling call
0709 12:10:17.467981 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.468825 (+   844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.469034 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37760}
I20260709 12:10:17.472187 10433 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:17.472797 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1181) took 2175 ms. Trace:
I20260709 12:10:17.473286 10433 rpcz_store.cc:276] 0709 12:10:15.297462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.471341 (+2173879us) service_pool.cc:224] Handling call
0709 12:10:17.471756 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.472548 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.472745 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.473737 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:17.461733 10434 rpcz_store.cc:276] 0709 12:10:15.297161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.459609 (+2162448us) service_pool.cc:224] Handling call
0709 12:10:17.460985 (+  1376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.475008 10433 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:17.475255 10430 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:17.475828 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1182) took 2178 ms. Trace:
I20260709 12:10:17.475991 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1152) took 2273 ms. Trace:
I20260709 12:10:17.476279 10433 rpcz_store.cc:276] 0709 12:10:15.297934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.474125 (+2176191us) service_pool.cc:224] Handling call
0709 12:10:17.474635 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.475490 (+   855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.475773 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.477628 12213 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:17.476449 10430 rpcz_store.cc:276] 0709 12:10:15.202301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.465073 (+2262772us) service_pool.cc:224] Handling call
0709 12:10:17.467997 (+  2924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.475718 (+  7721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.475905 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:17.481952 10438 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 89) took 2283 ms (client timeout 2142 ms). Trace:
I20260709 12:10:17.482035 10429 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:17.482471 10438 rpcz_store.cc:269] 0709 12:10:15.198907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.481574 (+2282667us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:17.481736 (+   162us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:17.493314 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:17.506480 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.509176 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:17.510613 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.516006 12235 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.516716 12235 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:16.933496 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.519269 12268 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:17.521862 12235 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:17.526183 12263 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.527890 12263 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:17.533237 12263 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:17.543910 12068 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:17.546895 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 626691us
I20260709 12:10:17.559019 12213 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:17.560197 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:17.562978 11213 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEARNER]: Advancing to term 4
I20260709 12:10:17.582402 12225 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:17.582523 10479 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.605s (SENT) [suppressed 41 similar messages]
I20260709 12:10:17.583365 12225 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:17.586333 12225 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:17.587424 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:17.583215 10447 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-deab11eaf55b4f3ebe48926a1e2afa38 received from {username='slave'} at 127.0.0.1:54450
I20260709 12:10:17.593559 10447 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-deab11eaf55b4f3ebe48926a1e2afa38 on tablet deab11eaf55b4f3ebe48926a1e2afa38 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:17.594738 10447 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.EndTabletCopySession from 127.0.0.1:54450 (request call id 86) took 2397 ms. Trace:
I20260709 12:10:17.596292 10447 rpcz_store.cc:276] 0709 12:10:15.197717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.582777 (+2385060us) service_pool.cc:224] Handling call
0709 12:10:17.594642 (+ 11865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.594628 12068 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:17.602370 11213 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:17.602916 11213 pending_rounds.cc:85] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 2
I20260709 12:10:17.603310 11213 pending_rounds.cc:107] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 12:10:17.603726 11213 raft_consensus.cc:2889] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:17.607484 10581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 1777) took 2113 ms (client timeout 1888 ms). Trace:
W20260709 12:10:17.608065 10581 rpcz_store.cc:269] 0709 12:10:15.494151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.688145 (+193994us) service_pool.cc:224] Handling call
0709 12:10:17.599632 (+1911487us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003dc054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:17.607243 (+  7611us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2549761920}
I20260709 12:10:17.618801 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1160) took 2349 ms. Trace:
I20260709 12:10:17.621165 10429 rpcz_store.cc:276] 0709 12:10:15.269726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.445744 (+2176018us) service_pool.cc:224] Handling call
0709 12:10:17.618553 (+172809us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b54002666d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:17.618707 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49824000}
I20260709 12:10:16.898993 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1434) took 1295 ms. Trace:
I20260709 12:10:17.636991 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1441) took 1239 ms. Trace:
I20260709 12:10:17.637873 10429 rpcz_store.cc:276] 0709 12:10:16.397908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.626529 (+1228621us) service_pool.cc:224] Handling call
0709 12:10:17.627065 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.627455 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:17.627920 (+   465us) log.cc:844] Serialized 14 byte log entry
0709 12:10:17.636361 (+  8441us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:17.636579 (+   218us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:17.636655 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.636744 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.636912 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1773,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":97,"thread_start_us":7804,"threads_started":1,"wal-append.queue_time_us":8243}
I20260709 12:10:17.633976 10578 rpcz_store.cc:276] 0709 12:10:15.603639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.598956 (+995317us) service_pool.cc:224] Handling call
0709 12:10:16.627586 (+ 28630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.713818 (+ 86232us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:16.783114 (+ 69296us) log.cc:844] Serialized 14 byte log entry
0709 12:10:16.897526 (+114412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.897599 (+    73us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:16.898468 (+   869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.898570 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.898756 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24913,"prepare.queue_time_us":22590,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":67,"raft.queue_time_us":79205,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":34904,"threads_started":1,"wal-append.queue_time_us":92750}
W20260709 12:10:16.868945 10505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 1780) took 1362 ms (client timeout 1504 ms). Trace:
W20260709 12:10:17.656509 10505 rpcz_store.cc:269] 0709 12:10:15.506496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.591144 (+ 84648us) service_pool.cc:224] Handling call
0709 12:10:16.868802 (+1277658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.658092 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:17.667748 10506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1241) took 2389 ms (client timeout 1673 ms). Trace:
I20260709 12:10:17.679819 12074 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:17.670821 12131 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 3.797s	user 0.009s	sys 0.008s
I20260709 12:10:17.681876 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.682714 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1382) took 4778 ms. Trace:
I20260709 12:10:17.682997 10429 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 2: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:17.684247 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.685027 10429 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:17.685321 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:17.679222 10506 rpcz_store.cc:269] 0709 12:10:15.278488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.278633 (+   145us) service_pool.cc:224] Handling call
0709 12:10:17.667258 (+2388625us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002c65d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:17.667488 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3096094592}
I20260709 12:10:17.687001 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:17.683207 10579 rpcz_store.cc:276] 0709 12:10:12.903769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.208249 (+304480us) service_pool.cc:224] Handling call
0709 12:10:13.208831 (+   582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:13.267682 (+ 58851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:13.270341 (+  2659us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:13.291735 (+ 21394us) log.cc:844] Serialized 26 byte log entry
0709 12:10:13.442029 (+150294us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b48000b19d8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f 00007f2c97af6f37 00007f2c97b0b55c 00007f2c97b0b508 00007f2c97b0b488 00007f2c97b0b443 00007f2c97b0a6ec 00007f2c96a34a96
0709 12:10:13.718508 (+276479us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:14.366093 (+647585us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b18001fc960. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c964d73c7 00007f2c92ae99a7 00007f2c92af90bf 00007f2c92af9038
0709 12:10:14.511044 (+144951us) log.cc:844] Serialized 14 byte log entry
0709 12:10:14.752636 (+241592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.756777 (+  4141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:15.891830 (+1135053us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b18001fc960. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:17.669903 (+1778073us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:17.682235 (+ 12332us) raft_consensus.cc:1685] finished
0709 12:10:17.682257 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.682469 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125219,"prepare.queue_time_us":774578,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":93208,"raft.run_cpu_time_us":2823,"raft.run_wall_time_us":182220,"spinlock_wait_cycles":138447104,"thread_start_us":587156,"threads_started":2,"wal-append.queue_time_us":493286}
I20260709 12:10:17.687618 12213 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:17.688330 12213 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.670050 10505 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:17.688503 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.689517 10429 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:17.691401 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:17.692324 10429 raft_consensus.cc:2368] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 4.
I20260709 12:10:17.692453 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:17.693630 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.688568 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.694336 10429 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:17.694728 10506 raft_consensus.cc:2393] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:17.695333 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:17.695727 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.689733 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:17.697077 10505 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.697188 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:17.698493 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.699134 10506 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:17.700037 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.700613 10506 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:17.701519 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:17.704216 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:17.704434 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.736496 10429 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:17.737759 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.738451 10429 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
I20260709 12:10:17.712908 12074 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.723498 10505 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:17.739356 12074 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:17.740110 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.741344 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.741652 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.742370 10429 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:17.742563 10505 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
W20260709 12:10:17.725150 10435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1190) took 2284 ms (client timeout 2063 ms). Trace:
I20260709 12:10:17.743691 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.743746 10435 rpcz_store.cc:269] 0709 12:10:15.440436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.724869 (+2284433us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:17.725034 (+   165us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:17.745124 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:17.723551 10432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1165) took 2436 ms (client timeout 1963 ms). Trace:
W20260709 12:10:17.745878 10432 rpcz_store.cc:269] 0709 12:10:15.287209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.723298 (+2436089us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:17.723467 (+   169us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:17.729074 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1155) took 2511 ms. Trace:
I20260709 12:10:17.746901 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.748946 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.749084 10432 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:17.750422 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.752388 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.753163 10432 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:17.747260 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.754868 10435 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:17.755365 10435 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:17.745167 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:17.746891 10436 rpcz_store.cc:276] 0709 12:10:15.217309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.467380 (+2250071us) service_pool.cc:224] Handling call
0709 12:10:17.467949 (+   569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.595575 (+127626us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:17.595610 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:17.595696 (+    86us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:17.595729 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.728632 (+132903us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540033d6d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:17.728717 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.728942 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28592,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":1130,"spinlock_wait_cycles":292413184}
I20260709 12:10:17.756630 10435 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:17.757550 10436 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:17.757932 10436 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:17.758069 10435 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.753237 10505 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:17.764106 12265 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:17.760648 10436 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:17.764742 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 43691.5us
I20260709 12:10:17.762329 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:17.765632 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.008s	sys 0.021s
I20260709 12:10:17.762810 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:17.766146 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.008s	sys 0.021s
I20260709 12:10:17.766213 10506 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:17.766556 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 933335us
W20260709 12:10:17.766594 10512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1242) took 2453 ms (client timeout 1839 ms). Trace:
I20260709 12:10:17.767271 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:17.771624 12263 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:17.767175 10512 rpcz_store.cc:269] 0709 12:10:15.313440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.324238 (+ 10798us) service_pool.cc:224] Handling call
0709 12:10:17.762440 (+2438202us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b5800003098. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:17.766514 (+  4074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30054272}
W20260709 12:10:17.769310 10479 leader_election.cc:336] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.009s (SENT)
I20260709 12:10:17.775486 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:17.754726 10432 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:17.778139 10511 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:10:17.776078 10512 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:17.781369 11950 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.780532 10511 pending_rounds.cc:85] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 1
I20260709 12:10:17.783479 10511 pending_rounds.cc:107] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:10:17.784056 10511 raft_consensus.cc:2889] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:17.781438 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1781) took 2274 ms. Trace:
I20260709 12:10:17.773301 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:17.785882 10512 rpcz_store.cc:276] 0709 12:10:15.506781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.775408 (+2268627us) service_pool.cc:224] Handling call
0709 12:10:17.775805 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.781121 (+  5316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.781354 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.793934 10512 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:17.794839 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1251) took 2112 ms. Trace:
I20260709 12:10:17.730932 12213 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:17.732679 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:17.794001 12235 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:17.797989 12213 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:17.765518 10436 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:17.800679 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 68191.8us
I20260709 12:10:17.802423 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:17.807250 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 271) took 3720 ms. Trace:
I20260709 12:10:16.851640 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1433) took 1227 ms. Trace:
I20260709 12:10:17.807920 11215 rpcz_store.cc:276] 0709 12:10:14.086702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.091059 (+  4357us) service_pool.cc:224] Handling call
0709 12:10:17.802029 (+3710970us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:17.802438 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.802954 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:17.805089 (+  2135us) log.cc:844] Serialized 14 byte log entry
0709 12:10:17.806558 (+  1469us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:17.806630 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:17.806684 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.806786 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.807007 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1329,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":50,"raft.queue_time_us":122,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":7208320,"thread_start_us":1357,"threads_started":2,"wal-append.queue_time_us":1081}
I20260709 12:10:17.810245 10582 rpcz_store.cc:276] 0709 12:10:15.603269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.477889 (+874620us) service_pool.cc:224] Handling call
0709 12:10:16.524747 (+ 46858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.570550 (+ 45803us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:16.608966 (+ 38416us) log.cc:844] Serialized 14 byte log entry
0709 12:10:16.765361 (+156395us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.807143 (+ 41782us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:16.820524 (+ 13381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.824399 (+  3875us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.830725 (+  6326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104394,"prepare.queue_time_us":36061,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":4781,"raft.queue_time_us":125016,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":13370,"thread_start_us":62694,"threads_started":1,"wal-append.queue_time_us":267987}
I20260709 12:10:17.810250 12235 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.814424 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:17.816051 10403 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
I20260709 12:10:17.816422 10403 leader_election.cc:403] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:17.818001 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:17.826853 12222 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.828446 12222 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:17.829962 12114 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:17.830494 12114 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:17.830941 12114 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:17.830729 12222 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:17.839219 11219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 269) took 2356 ms. Trace:
I20260709 12:10:17.839684 10435 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:17.842970 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:17.844525 12274 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:17.845343 12274 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:18.172607 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:18.039625 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.846485 12114 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:18.158164 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50371.8us
I20260709 12:10:17.839686 11219 rpcz_store.cc:276] 0709 12:10:15.482731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.483035 (+   304us) service_pool.cc:224] Handling call
0709 12:10:16.105023 (+621988us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:16.105360 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.223968 (+118608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.224062 (+    94us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.224166 (+   104us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:16.224242 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.838756 (+1614514us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540018fbd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:17.838803 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.839002 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253738624}
I20260709 12:10:17.795384 10512 rpcz_store.cc:276] 0709 12:10:15.682582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.792899 (+2110317us) service_pool.cc:224] Handling call
0709 12:10:17.793444 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.794477 (+  1033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.794749 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:18.264156 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.856637 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1246) took 2488 ms. Trace:
I20260709 12:10:17.831982 12121 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:17.855451 12183 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.002s	sys 0.007s
I20260709 12:10:18.085925 12272 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:17.947861 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 89) took 2459 ms. Trace:
I20260709 12:10:17.832952 12265 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:17.853745 12267 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:10:18.241925 12082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.414s	user 0.001s	sys 0.015s
I20260709 12:10:17.854542 11215 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:10:18.148902 11990 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:18.275413 11990 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:17.954427 12229 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 2.362s	user 0.000s	sys 0.009s
I20260709 12:10:18.273972 11930 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:18.270275 12183 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.369s	user 0.004s	sys 0.010s
I20260709 12:10:17.852108 12213 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:18.276981 11930 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:18.283056 11930 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:18.284230 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51396.5us
I20260709 12:10:18.265291 10504 rpcz_store.cc:276] 0709 12:10:15.315466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.318327 (+  2861us) service_pool.cc:224] Handling call
0709 12:10:17.742001 (+2423674us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5800003098. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:17.742326 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.742673 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:17.743151 (+   478us) log.cc:844] Serialized 14 byte log entry
0709 12:10:17.803225 (+ 60074us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:17.803301 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:17.803355 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.803434 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.803574 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2155,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":118,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"spinlock_wait_cycles":30963072,"thread_start_us":59482,"threads_started":1,"wal-append.queue_time_us":51167}
I20260709 12:10:18.283308 12272 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:18.274179 12082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.447s	user 0.001s	sys 0.016s
I20260709 12:10:17.008394 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:18.289309 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1435) took 1300 ms. Trace:
W20260709 12:10:18.290868 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:18.291308 12272 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:18.272645 10505 rpcz_store.cc:276] 0709 12:10:15.487828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.775267 (+2287439us) service_pool.cc:224] Handling call
0709 12:10:17.775743 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.776040 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:17.777634 (+  1594us) log.cc:844] Serialized 14 byte log entry
0709 12:10:17.891301 (+113667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:17.947184 (+ 55883us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:17.947287 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.947415 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.947682 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":200,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":7754,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":36480,"thread_start_us":74246,"threads_started":1}
I20260709 12:10:18.298221 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1254) took 2437 ms. Trace:
I20260709 12:10:18.300361 12114 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:18.290580 10587 rpcz_store.cc:276] 0709 12:10:16.989078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.007988 (+ 18910us) service_pool.cc:224] Handling call
0709 12:10:18.289189 (+1281201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:17.846779 12274 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:18.279323 12213 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:18.303071 12213 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.298666 10505 rpcz_store.cc:276] 0709 12:10:15.861054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.296943 (+2435889us) service_pool.cc:224] Handling call
0709 12:10:18.297413 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.297719 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.297761 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:18.297839 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:18.297882 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.297952 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.298122 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:18.329077 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:18.341534 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1253) took 2486 ms. Trace:
I20260709 12:10:18.342166 10504 rpcz_store.cc:276] 0709 12:10:15.855474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.304111 (+2448637us) service_pool.cc:224] Handling call
0709 12:10:18.330672 (+ 26561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.341040 (+ 10368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.341075 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:18.341130 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:18.341170 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.341251 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.341432 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:18.344448 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1249) took 2859 ms. Trace:
I20260709 12:10:18.344996 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:18.345917 10505 raft_consensus.cc:606] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:18.344945 10506 rpcz_store.cc:276] 0709 12:10:15.485477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.770364 (+2284887us) service_pool.cc:224] Handling call
0709 12:10:17.770940 (+   576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.771380 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.771410 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:17.772167 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 12:10:17.847390 (+ 75223us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:17.847433 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.847506 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:18.297356 (+449850us) spinlock_profiling.cc:243] Waited 5.02 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:18.344029 (+ 46673us) raft_consensus.cc:1685] finished
0709 12:10:18.344053 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.344340 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11548032,"thread_start_us":68484,"threads_started":1,"wal-append.queue_time_us":509123}
I20260709 12:10:18.346369 10505 raft_consensus.cc:614] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:18.347754 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1255) took 2056 ms. Trace:
I20260709 12:10:18.348313 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1257) took 1619 ms. Trace:
I20260709 12:10:18.348795 10505 rpcz_store.cc:276] 0709 12:10:16.729333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.347400 (+1618067us) service_pool.cc:224] Handling call
0709 12:10:18.347732 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.347944 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.347972 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:18.348026 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:18.348064 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.348127 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.348259 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:18.376893 10504 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:18.386824 10504 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:18.348117 10506 rpcz_store.cc:276] 0709 12:10:16.383728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.346763 (+1963035us) service_pool.cc:224] Handling call
0709 12:10:18.347134 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.347362 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.347401 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:18.347463 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:18.347499 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.347554 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.347677 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:18.409921 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08001d10e0 after lost signal to thread 9264
I20260709 12:10:18.348925 11990 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:18.442749 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6ce0 after lost signal to thread 10370
W20260709 12:10:18.444921 10459 debug-util.cc:398] Leaking SignalData structure 0x7b080039ab60 after lost signal to thread 10462
W20260709 12:10:18.445309 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08003abde0 after lost signal to thread 10463
I20260709 12:10:18.447084 10506 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:18.454171 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:18.455375 11930 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:18.455821 11930 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:18.456228 11930 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.477260 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:18.500080 12265 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:18.501564 12265 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:18.517580 11990 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:18.518661 11990 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:18.519363 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:18.521917 11990 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:18.522248 10506 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:18.541994 12183 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:18.573208 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:18.574420 10504 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:18.606058 10403 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.722s (SENT)
I20260709 12:10:18.607025 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:18.655143 12265 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:18.625427 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1258) took 1895 ms. Trace:
I20260709 12:10:18.656545 12128 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:18.656989 10505 rpcz_store.cc:276] 0709 12:10:16.729839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.350319 (+1620480us) service_pool.cc:224] Handling call
0709 12:10:18.350739 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.624740 (+274001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.624792 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:18.624873 (+    81us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:18.624937 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.625048 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.625290 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:18.657409 12128 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:18.660820 12128 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:18.668071 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62076.7us
W20260709 12:10:18.669304 10403 leader_election.cc:336] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.726s (SENT)
I20260709 12:10:18.669967 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:18.675489 12265 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:18.696534 12128 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:18.697822 12128 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:18.701100 12128 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:18.712225 10403 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.733s (SENT)
W20260709 12:10:18.739117 10403 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.733s (SENT)
W20260709 12:10:18.740571 10403 leader_election.cc:336] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.740s (SENT)
I20260709 12:10:18.741096 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:18.750095 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:18.790469 12265 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:18.792069 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51550.8us
W20260709 12:10:18.824384 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:18.766100 10504 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:18.785931 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 1793) took 3243 ms. Trace:
W20260709 12:10:18.843768 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:18.864538 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 351390us
I20260709 12:10:18.502414 11930 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:18.866807 10512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.013s	sys 0.006s
W20260709 12:10:18.871843 10512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.006s
W20260709 12:10:18.870956 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:18.839480 12213 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.536s	user 0.006s	sys 0.007s
I20260709 12:10:18.828933 10584 rpcz_store.cc:276] 0709 12:10:15.542795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.451627 (+908832us) service_pool.cc:224] Handling call
0709 12:10:16.483327 (+ 31700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:16.495307 (+ 11980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.495348 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:16.556755 (+ 61407us) log.cc:844] Serialized 26 byte log entry
0709 12:10:16.684256 (+127501us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:16.684318 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.684426 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:16.877256 (+192830us) raft_consensus.cc:1685] finished
0709 12:10:16.877300 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.785780 (+1908480us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":72662,"threads_started":1,"wal-append.queue_time_us":302185,"wal-append.run_cpu_time_us":16572,"wal-append.run_wall_time_us":1494260}
I20260709 12:10:18.872952 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1252) took 3074 ms. Trace:
W20260709 12:10:18.864951 12267 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
W20260709 12:10:18.878366 10403 leader_election.cc:336] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.871s (SENT)
I20260709 12:10:18.873878 11991 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:18.874286 12213 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:18.893018 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:18.897730 10403 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.907s (SENT)
I20260709 12:10:18.880767 12267 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3.
W20260709 12:10:18.899973 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:18.908475 10435 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:18.874876 10512 rpcz_store.cc:276] 0709 12:10:15.798362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.270594 (+2472232us) service_pool.cc:224] Handling call
0709 12:10:18.274591 (+  3997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.292237 (+ 17646us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:18.292572 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 12:10:18.872487 (+579915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:18.872565 (+    78us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:18.872632 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.872711 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:18.872873 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299799,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":120,"thread_start_us":579438,"threads_started":1}
I20260709 12:10:18.910321 10432 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:18.910733 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:18.917523 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:18.910513 11215 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:18.919047 12265 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:18.920372 10437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1188) took 3515 ms (client timeout 2058 ms). Trace:
I20260709 12:10:18.918623 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:18.923259 10429 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:18.925573 10435 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:10:18.920811 10437 rpcz_store.cc:269] 0709 12:10:15.405418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.920129 (+3514711us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:18.920293 (+   164us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:18.927537 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:18.928498 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:18.929503 10432 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:18.938401 10403 leader_election.cc:336] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.949s (SENT)
I20260709 12:10:18.938963 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:18.940821 10403 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 2.104s (SENT)
I20260709 12:10:18.941260 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:18.944234 12213 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:18.944942 12213 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:18.945616 12213 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:18.923632 10429 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:18.947088 10429 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:18.948649 10429 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:18.949918 11177 connection.cc:256] RPC call timeout handler was delayed by -1.2620201830000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 12:10:18.950368 11177 connection.cc:256] RPC call timeout handler was delayed by -1.2620201830000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 12:10:18.950639 11177 connection.cc:256] RPC call timeout handler was delayed by -1.2096692339998754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
I20260709 12:10:18.951851 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:18.952652 10432 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:18.952771 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:18.953466 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:18.954649 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:18.936802 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:18.959036 10432 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:18.960969 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:18.964991 10432 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:10:18.966147 12057 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.131s	user 0.025s	sys 0.012s
I20260709 12:10:18.968859 12274 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:18.969256 12292 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:18.970181 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:18.970156 12292 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:18.974561 11906 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:18.979466 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:18.980098 10435 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
W20260709 12:10:18.967185 12057 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.132s	user 0.025s	sys 0.012s
I20260709 12:10:18.988493 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:18.989208 10512 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.994316 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:18.996135 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:18.997584 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 268) took 4761 ms. Trace:
I20260709 12:10:18.998001 11221 rpcz_store.cc:276] 0709 12:10:14.236123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.236311 (+   188us) service_pool.cc:224] Handling call
0709 12:10:14.255764 (+ 19453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:14.774301 (+518537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.997088 (+4222787us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54004588d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:18.997367 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191052544}
I20260709 12:10:19.001492 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1244) took 3687 ms. Trace:
I20260709 12:10:19.001938 10513 rpcz_store.cc:276] 0709 12:10:15.314533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.316691 (+  2158us) service_pool.cc:224] Handling call
0709 12:10:15.498172 (+181481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:15.558377 (+ 60205us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:15.573520 (+ 15143us) log.cc:844] Serialized 14 byte log entry
0709 12:10:16.482251 (+908731us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.482374 (+   123us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:16.482535 (+   161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.001172 (+2518637us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b54000f8cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:19.001242 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.001411 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15348,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":58,"raft.queue_time_us":470,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":267,"spinlock_wait_cycles":137888128,"thread_start_us":904719,"threads_started":1,"wal-append.queue_time_us":1025499,"wal-append.run_cpu_time_us":1808,"wal-append.run_wall_time_us":1001832}
I20260709 12:10:19.003542 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.004503 10437 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.005982 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.007520 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.008239 10437 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:19.008427 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.013665 12297 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:19.019233 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:19.019101 12288 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:19.020610 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.021893 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.022534 10436 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:19.023296 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.023649 11950 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.024142 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.024500 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.024245 11950 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.025606 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.026721 11950 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:19.029904 12292 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.011729 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.026782 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.034885 10435 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.040829 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.042898 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.007982 12295 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.043547 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.043596 12295 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:19.044435 12295 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.044993 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.046252 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.049822 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.050902 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.009384 12298 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:19.055531 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:19.055976 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45770.5us
I20260709 12:10:19.056305 10436 raft_consensus.cc:2368] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:19.057277 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.057958 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:19.058665 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.058871 10512 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
W20260709 12:10:19.059369 12268 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.501s	user 0.015s	sys 0.024s
W20260709 12:10:19.059762 12268 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.501s	user 0.015s	sys 0.024s
I20260709 12:10:19.062006 10437 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.062762 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:19.070348 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.072962 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.073225 11906 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:19.073766 10436 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:19.074796 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.071961 12295 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.062908 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.079722 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.080034 10437 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.080662 10435 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.081056 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.075709 10436 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:19.081568 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.082536 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.082825 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.087046 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.088544 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.058950 10429 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:19.097461 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.090844 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.098484 10429 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:19.103335 10432 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:19.105077 10435 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.105185 12265 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:19.106635 12265 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.107059 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.108071 10435 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.113040 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.114440 10436 raft_consensus.cc:2410] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:19.101581 11906 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.118248 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.065346 12298 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.121951 10435 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:19.123368 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:19.126770 12267 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:19.126231 11906 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.127416 12267 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:19.128566 12267 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:19.129573 11906 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:19.126300 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "448b86aa128243e48bed5f471ada874f"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:19.142696 10436 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:19.117898 12265 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.143357 10436 pending_rounds.cc:85] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:19.145843 10436 pending_rounds.cc:107] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 12:10:19.146718 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.146227 10436 raft_consensus.cc:2889] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:19.142139 10429 raft_consensus.cc:606] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:19.151077 12298 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:19.131623 12295 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.151600 10429 raft_consensus.cc:614] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:19.144404 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:19.139287 11930 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:19.155563 10435 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:19.142020 11906 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:19.157353 12308 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:19.157248 11906 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.157910 12308 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:19.165503 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:19.166674 10506 raft_consensus.cc:2410] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:10:19.168316 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:17.376348 11217 rpcz_store.cc:276] 0709 12:10:14.372885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.465520 (+ 92635us) service_pool.cc:224] Handling call
0709 12:10:16.625741 (+2160221us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:16.626014 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.626583 (+   569us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:16.779349 (+152766us) log.cc:844] Serialized 14 byte log entry
0709 12:10:16.996781 (+217432us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:17.037766 (+ 40985us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:17.116840 (+ 79074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.116944 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.117185 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":353390,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":75,"raft.queue_time_us":709,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"spinlock_wait_cycles":691208576,"thread_start_us":326280,"threads_started":2,"wal-append.queue_time_us":607711}
I20260709 12:10:19.172770 10436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48980
I20260709 12:10:19.173337 10436 raft_consensus.cc:606] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership
I20260709 12:10:19.173650 10436 raft_consensus.cc:614] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:19.172359 12220 raft_consensus.cc:1068] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }.
  Pending config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.175879 10429 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:10:19.182029 11950 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:19.172194 11906 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.183333 12074 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.183531 12241 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:19.184984 12235 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:19.186901 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.208822 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:19.213294 10506 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:19.214836 12241 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:19.213901 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.149425 12268 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:10:19.215299 12241 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:19.217517 12268 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:19.217900 12268 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:19.218010 12241 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.218948 12268 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.195539 11930 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:19.220779 11930 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:19.221197 11218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 282) took 1665 ms (client timeout 1665 ms). Trace:
W20260709 12:10:19.221702 11218 rpcz_store.cc:269] 0709 12:10:17.555668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.220964 (+1665296us) service_pool.cc:211] Skipping call since client already timed out
0709 12:10:19.221099 (+   135us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:10:19.223495 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.220273 12135 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:19.219517 12241 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.226070 12135 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:19.227252 12135 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.159094 12308 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:19.229890 12228 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.233309 11930 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:19.228622 12135 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.234202 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.237731 11950 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:19.240914 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.228643 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:19.244622 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.245461 10582 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.243592 11930 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:19.245836 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.246615 10429 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.247591 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.248209 10580 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.249781 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.250459 10584 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
W20260709 12:10:19.251689 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64882.9us
I20260709 12:10:19.252512 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:19.255162 10434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.765s	user 0.013s	sys 0.023s
I20260709 12:10:19.255256 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:19.255774 10434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.766s	user 0.013s	sys 0.023s
I20260709 12:10:19.256479 10506 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:19.257603 12223 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:19.257787 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1183) took 3959 ms. Trace:
I20260709 12:10:19.255378 12139 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.258546 12139 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:19.258935 10479 leader_election.cc:400] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:19.251140 10432 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:19.273016 12213 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:19.273443 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.260231 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.261106 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:19.261979 10508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 1776) took 3818 ms (client timeout 1947 ms). Trace:
I20260709 12:10:19.259069 12139 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:19.269654 12270 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.687s	user 0.011s	sys 0.013s
I20260709 12:10:19.284828 10578 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:19.296294 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.298578 10587 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:19.299134 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.272257 10579 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:19.300850 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.301031 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.301662 10578 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:19.301877 11216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 271) took 2471 ms (client timeout 2203 ms). Trace:
I20260709 12:10:19.302193 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.282042 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.296402 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:19.296258 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 105914us
I20260709 12:10:19.285003 10432 pending_rounds.cc:85] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 5
I20260709 12:10:19.303922 10584 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:19.302446 11216 rpcz_store.cc:269] 0709 12:10:16.855570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.871253 (+ 15683us) service_pool.cc:224] Handling call
0709 12:10:19.301771 (+2430518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:19.305464 10432 pending_rounds.cc:107] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 12:10:19.307781 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48958.3us
I20260709 12:10:19.308300 10432 raft_consensus.cc:2889] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:19.304440 10583 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:19.308665 10432 pending_rounds.cc:107] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.7
W20260709 12:10:19.275137 10508 rpcz_store.cc:269] 0709 12:10:15.443664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.443917 (+   253us) service_pool.cc:224] Handling call
0709 12:10:19.228274 (+3784357us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b5800003098. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:19.261850 (+ 33576us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16400256}
I20260709 12:10:19.310684 10432 raft_consensus.cc:2889] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:19.282553 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.316020 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:19.319478 12250 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 2.397s	user 0.001s	sys 0.042s
I20260709 12:10:19.321381 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1775) took 3878 ms. Trace:
I20260709 12:10:19.321755 10510 rpcz_store.cc:276] 0709 12:10:15.443293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.443823 (+   530us) service_pool.cc:224] Handling call
0709 12:10:16.033837 (+590014us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b5800003098. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:16.034112 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:16.034693 (+   581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.034739 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:16.196473 (+161734us) log.cc:844] Serialized 26 byte log entry
0709 12:10:16.538408 (+341935us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:16.679876 (+141468us) log.cc:844] Serialized 14 byte log entry
0709 12:10:16.716375 (+ 36499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.716514 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.318787 (+2602273us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:19.320936 (+  2149us) raft_consensus.cc:1685] finished
0709 12:10:19.320958 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.321122 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34345,"prepare.queue_time_us":245147,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":16359,"raft.queue_time_us":41286,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":8839,"spinlock_wait_cycles":172096512,"thread_start_us":398499,"threads_started":2,"wal-append.queue_time_us":722334}
I20260709 12:10:19.315677 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.324644 10581 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.271659 12274 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:10:19.332083 12307 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:19.333204 12299 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:19.336581 10584 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:19.340975 12241 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:19.340766 12307 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.344309 12299 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:19.344756 12299 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:19.346241 12261 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 1.931s	user 0.013s	sys 0.030s
I20260709 12:10:19.346837 12299 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.348479 12299 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:19.349859 12241 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:19.351265 12307 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:19.352975 10429 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
W20260709 12:10:19.353525 12269 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.817s	user 0.014s	sys 0.013s
I20260709 12:10:19.302903 10579 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:19.354720 10580 raft_consensus.cc:2410] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:19.374557 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1184) took 4076 ms. Trace:
I20260709 12:10:19.365433 10581 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:19.355793 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.368383 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:19.420671 10587 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:19.412583 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.415369 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1185) took 4116 ms. Trace:
I20260709 12:10:19.426298 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:19.427410 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59161.3us
I20260709 12:10:19.429217 12295 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:19.429883 12295 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:19.428704 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.356140 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.432744 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:19.374158 12271 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.838s	user 0.014s	sys 0.016s
I20260709 12:10:19.425802 10430 rpcz_store.cc:276] 0709 12:10:15.298885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.479479 (+2180594us) service_pool.cc:224] Handling call
0709 12:10:17.480073 (+   594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.483438 (+  3365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.483473 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:17.487889 (+  4416us) log.cc:844] Serialized 26 byte log entry
0709 12:10:17.534002 (+ 46113us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:17.576936 (+ 42934us) log.cc:844] Serialized 14 byte log entry
0709 12:10:17.620323 (+ 43387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.621457 (+  1134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.352721 (+1731264us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:19.414915 (+ 62194us) raft_consensus.cc:1685] finished
0709 12:10:19.415083 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.415272 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":86824,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":19557,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":280,"spinlock_wait_cycles":401958656,"thread_start_us":130195,"threads_started":3,"wal-append.queue_time_us":47957}
I20260709 12:10:19.364750 11930 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:19.430878 12068 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.830s	user 0.010s	sys 0.024s
W20260709 12:10:19.436652 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 128006us
I20260709 12:10:19.438424 10584 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.411502 10433 rpcz_store.cc:276] 0709 12:10:15.298592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.477598 (+2179006us) service_pool.cc:224] Handling call
0709 12:10:17.478114 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.478567 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.478598 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:17.487927 (+  9329us) log.cc:844] Serialized 26 byte log entry
0709 12:10:17.558008 (+ 70081us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:17.558063 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.558202 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.268859 (+1710657us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:19.364019 (+ 95160us) raft_consensus.cc:1685] finished
0709 12:10:19.364041 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.374472 (+ 10431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298653952,"thread_start_us":66617,"threads_started":1,"wal-append.queue_time_us":93259}
I20260709 12:10:19.440968 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.441460 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1186) took 4111 ms. Trace:
I20260709 12:10:19.445658 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.446355 10581 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:19.447372 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.448001 10581 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:19.449362 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.445111 10438 rpcz_store.cc:276] 0709 12:10:15.331141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.484429 (+2153288us) service_pool.cc:224] Handling call
0709 12:10:17.489798 (+  5369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.490184 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.490222 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:17.490893 (+   671us) log.cc:844] Serialized 26 byte log entry
0709 12:10:17.529399 (+ 38506us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:17.529449 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.529555 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.363609 (+1834054us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:19.441117 (+ 77508us) raft_consensus.cc:1685] finished
0709 12:10:19.441141 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.441378 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399983872,"thread_start_us":37480,"threads_started":1,"wal-append.queue_time_us":44490}
I20260709 12:10:19.451762 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.438977 12295 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.442198 10582 raft_consensus.cc:2410] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:19.444207 12068 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.844s	user 0.011s	sys 0.024s
I20260709 12:10:19.453653 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.438674 10585 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:19.459111 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:19.461689 12135 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.462136 12135 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:19.462491 12135 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.463976 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.464949 10582 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
W20260709 12:10:19.457379 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 123284us
I20260709 12:10:19.471253 10583 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:19.477293 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.477764 10580 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:19.468007 10579 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:19.258296 10434 rpcz_store.cc:276] 0709 12:10:15.298293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.474802 (+2176509us) service_pool.cc:224] Handling call
0709 12:10:17.475276 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.478758 (+  3482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.478787 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:17.489769 (+ 10982us) log.cc:844] Serialized 26 byte log entry
0709 12:10:19.224405 (+1734636us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c8f9d50ac 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b
0709 12:10:19.256372 (+ 31967us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:19.256417 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.256492 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.257287 (+   795us) raft_consensus.cc:1685] finished
0709 12:10:19.257456 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.257701 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1182,"raft.queue_time_us":3448,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":227,"spinlock_wait_cycles":509355904,"thread_start_us":1766109,"threads_started":1,"wal-append.queue_time_us":1765542}
I20260709 12:10:19.478996 10581 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:19.483474 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.486737 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:19.481149 12266 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 2.044s	user 0.005s	sys 0.021s
I20260709 12:10:19.488072 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.489542 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.490478 10579 raft_consensus.cc:2410] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:19.469062 12068 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:19.467922 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.493563 10585 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:19.493997 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:19.500097 10578 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:19.503484 10504 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:19.508745 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1228) took 4414 ms. Trace:
I20260709 12:10:19.509449 10585 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:19.506735 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.500612 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:19.510164 10507 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 2.884s	user 0.030s	sys 0.018s
I20260709 12:10:19.502753 12241 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.511134 12241 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:19.511356 10513 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.511557 12241 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.512405 10508 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:19.513929 12068 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.509173 10511 rpcz_store.cc:276] 0709 12:10:15.088431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.090636 (+  2205us) service_pool.cc:224] Handling call
0709 12:10:15.091035 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.785984 (+2694949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.495778 (+1709794us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b540036f454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:19.502737 (+  6959us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6178323072}
I20260709 12:10:19.494549 12139 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.517505 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:19.519308 12139 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.509976 10585 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:19.532128 10580 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
W20260709 12:10:19.532842 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:19.528527 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.534577 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.535918 10587 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.537074 12057 raft_consensus.cc:2764] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:10:19.540657 12263 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.541144 12263 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:19.541527 12263 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.542655 12077 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:19.543166 12057 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.543776 12077 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.543850 12057 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.544461 12077 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.549157 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.536931 10507 raft_consensus.cc:2410] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:19.552201 10507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 1783) took 4044 ms (client timeout 3108 ms). Trace:
I20260709 12:10:19.533699 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:19.552596 10507 rpcz_store.cc:269] 0709 12:10:15.507489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.565661 (+1058172us) service_pool.cc:224] Handling call
0709 12:10:19.552088 (+2986427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:19.553500 10432 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.554261 10509 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.554397 12243 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.555318 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.555825 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.556039 11212 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:19.551095 12223 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:19.557078 12223 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:19.557919 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.559000 10434 raft_consensus.cc:2410] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:19.558226 12223 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.559940 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.560817 10434 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.562182 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.564983 12057 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:19.565253 12277 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.928s	user 0.019s	sys 0.008s
I20260709 12:10:19.566044 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.563135 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.549288 12077 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:19.567531 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.568302 10430 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:19.562834 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.569465 10430 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:19.560065 12223 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.571019 10433 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:19.566134 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.570003 10431 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:10:19.558141 12254 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 2.637s	user 0.010s	sys 0.028s
I20260709 12:10:19.563931 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.582875 10432 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:19.589324 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:19.591720 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.550788 10585 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.595669 10585 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:19.603691 12135 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:19.606900 10332 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 105) took 11691 ms (client timeout 15000 ms). Trace:
I20260709 12:10:19.609163 10435 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:19.614066 10508 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:10:19.620947 10552 connection.cc:256] RPC call timeout handler was delayed by -1.0393334509999477s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 12:10:19.624001 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:19.617738 10584 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:19.626959 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
W20260709 12:10:19.632530 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:19.637458 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1248) took 4321 ms. Trace:
I20260709 12:10:19.635782 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.638036 10509 rpcz_store.cc:276] 0709 12:10:15.316484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.317115 (+   631us) service_pool.cc:224] Handling call
0709 12:10:16.870404 (+1553289us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b5800003098. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:19.553945 (+2683541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:19.554649 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.614900 (+ 60251us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b54003fd9d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:19.637350 (+ 22450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6173083264}
W20260709 12:10:19.649101 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64609.3us
I20260709 12:10:19.608455 10434 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.608505 12135 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.608088 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 3 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 4 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:19.612671 12263 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:19.674219 10478 leader_election.cc:336] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.507s (SENT)
I20260709 12:10:19.676095 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.668576 10336 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 3, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:19.651515 12299 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:19.675563 10553 leader_election.cc:336] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.061s (SENT)
W20260709 12:10:19.667640 12316 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:10:19.678422 10336 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:19.678676 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.652431 12228 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:10:19.670639 12317 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 12:10:19.688898 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8440 after lost signal to thread 10537
I20260709 12:10:19.687196 10336 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:19.695263 12224 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:10:19.697880 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.699424 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.699971 11218 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:19.700284 12171 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:19.701001 10430 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.701700 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.701023 12171 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:19.705298 12268 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:19.705843 12263 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:19.706996 12171 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.707540 12171 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:19.708061 12171 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.705878 10585 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:10:19.624697 10332 rpcz_store.cc:269] 0709 12:10:07.945530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.945851 (+   321us) service_pool.cc:224] Handling call
0709 12:10:19.606663 (+11660812us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:14.164352 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:14.347235 (+182883us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:14.351649 (+  4414us) write_op.cc:435] Acquired schema lock
0709 12:10:14.356163 (+  4514us) tablet.cc:662] Decoding operations
0709 12:10:14.565583 (+209420us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:14.565689 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:14.565773 (+    84us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:10:14.567214 (+  1441us) tablet.cc:701] Row locks acquired
0709 12:10:14.567256 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:10:14.567558 (+   302us) write_op.cc:270] Start()
0709 12:10:14.567768 (+   210us) write_op.cc:276] Timestamp: P: 1783599014567474 usec, L: 0
0709 12:10:14.567816 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:10:14.569329 (+  1513us) log.cc:844] Serialized 3231 byte log entry
0709 12:10:14.928196 (+358867us) op_driver.cc:464] REPLICATION: finished
0709 12:10:14.990008 (+ 61812us) write_op.cc:301] APPLY: starting
0709 12:10:14.990129 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:14.990590 (+   461us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:15.006290 (+ 15700us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:15.010033 (+  3743us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:15.012743 (+  2710us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:10:15.012849 (+   106us) write_op.cc:312] APPLY: finished
0709 12:10:15.014118 (+  1269us) log.cc:844] Serialized 65 byte log entry
0709 12:10:15.015612 (+  1494us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:15.015840 (+   228us) write_op.cc:454] Released schema lock
0709 12:10:15.015944 (+   104us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:15.016559 (+   615us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:15.016613 (+    54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":468803,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":54428,"apply.run_cpu_time_us":6256,"apply.run_wall_time_us":26513,"num_ops":8,"prepare.queue_time_us":201857,"prepare.run_cpu_time_us":160521,"prepare.run_wall_time_us":610589,"raft.queue_time_us":50727,"raft.run_cpu_time_us":38048,"raft.run_wall_time_us":48201,"replication_time_us":359471,"thread_start_us":386003,"threads_started":4,"wal-append.queue_time_us":269797,"wal-append.run_cpu_time_us":15194,"wal-append.run_wall_time_us":1174927}]]}
I20260709 12:10:19.712574 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:19.684070 12320 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:10:19.714713 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47622.1us
I20260709 12:10:19.718375 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.723984 12288 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:19.709939 12299 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:19.724447 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.725356 12299 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:19.728091 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62169.4us
I20260709 12:10:19.719650 12074 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:19.729698 10553 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.388s (SENT)
I20260709 12:10:19.733449 10585 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:19.730851 10553 leader_election.cc:336] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.474s (SENT)
W20260709 12:10:19.720479 10459 debug-util.cc:398] Leaking SignalData structure 0x7b0800378440 after lost signal to thread 10611
I20260709 12:10:19.738448 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.740038 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.743677 10585 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:19.722325 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50765.1us
I20260709 12:10:19.742887 12292 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.745399 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.714196 12243 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:19.748198 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.731292 12295 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.755280 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.755985 10581 raft_consensus.cc:2376] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4. Re-sending same reply.
W20260709 12:10:19.745512 10459 debug-util.cc:398] Leaking SignalData structure 0x7b0800361100 after lost signal to thread 10612
I20260709 12:10:19.758916 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.752804 12295 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:19.763078 12295 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.746568 12292 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:19.767798 12292 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.767884 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:19.760334 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.745164 11906 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:10:19.784888 11888 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:19.742904 12276 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:10:19.789529 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.793378 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:19.804859 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.805550 10437 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:19.799124 12266 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:10:19.808746 10430 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:19.750877 12263 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:19.752562 12135 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.815237 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:19.816025 10585 raft_consensus.cc:2376] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3. Re-sending same reply.
I20260709 12:10:19.823654 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:19.824800 10581 raft_consensus.cc:2410] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:19.832688 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:19.833093 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.826018 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:19.839555 10432 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.832136 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.838160 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:19.844197 10438 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.797633 12068 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:19.746681 12220 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:10:19.862005 10583 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:19.847875 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:19.848417 11218 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:19.808765 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.868599 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.826083 12135 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:19.831241 12295 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:19.809715 12241 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:19.824715 12223 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
W20260709 12:10:19.926208 10478 leader_election.cc:336] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.579s (SENT)
I20260709 12:10:19.831938 12074 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
W20260709 12:10:19.905866 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 97819.8us
I20260709 12:10:19.838762 12139 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 12:10:19.924424 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55822.8us
W20260709 12:10:19.924655 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69655.4us
I20260709 12:10:19.847685 12243 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:19.927268 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:19.838410 12213 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:19.875406 12135 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:19.869282 10459 debug-util.cc:398] Leaking SignalData structure 0x7b0800377600 after lost signal to thread 11248
W20260709 12:10:19.883249 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 124933us
I20260709 12:10:19.873827 12068 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:19.940323 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:19.870676 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:19.984577 11212 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:19.987206 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:19.922925 12307 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:19.904265 11216 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:19.936370 12228 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:10:19.941931 12243 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:20.061556 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.061844 12243 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:20.066277 12326 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:10:20.066638 12319 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:19.950181 10581 pending_rounds.cc:85] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 4
I20260709 12:10:20.068370 10581 pending_rounds.cc:107] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:10:19.986564 12102 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 12:10:20.070022 12298 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:20.070017 12102 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:20.070914 12102 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:20.072639 12102 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:20.032758 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49829.2us
I20260709 12:10:20.075736 12342 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.076599 12241 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.070974 12298 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.053398 12276 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:20.079763 12276 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:20.080446 12276 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.082176 12298 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:19.903915 11907 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.115s
I20260709 12:10:20.074652 12102 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.084335 11907 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.084903 11907 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:20.041864 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 115713us
I20260709 12:10:20.082872 10438 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.053576 10434 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.090945 10431 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:20.091719 10429 raft_consensus.cc:606] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.092278 10429 raft_consensus.cc:614] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:20.068751 10581 raft_consensus.cc:2889] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:20.095643 12292 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.078965 12295 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.037866 11217 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:10:20.037160 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 97073.2us
I20260709 12:10:20.113399 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.113236 12333 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:20.115046 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:19.981506 12068 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:20.048296 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:20.116079 12265 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:20.116504 12068 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:20.037103 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 306262us
I20260709 12:10:20.116909 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.117415 12265 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:20.117945 12265 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:20.118870 12265 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.101967 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.119668 10585 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:20.098507 10478 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.686s (SENT)
I20260709 12:10:20.120662 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:20.122902 10478 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.738s (SENT)
I20260709 12:10:20.123462 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.110852 12307 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:20.086251 11907 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.125240 12307 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:20.118322 10553 leader_election.cc:336] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.799s (SENT)
I20260709 12:10:20.127494 12224 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:20.128494 10553 connection.cc:256] RPC call timeout handler was delayed by -3.2177146799999718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33580000000000004s for a response
W20260709 12:10:20.131424 10553 connection.cc:256] RPC call timeout handler was delayed by -2.9925645309999709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
W20260709 12:10:20.132164 10553 connection.cc:256] RPC call timeout handler was delayed by -2.9547645310001371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 12:10:20.132527 10553 connection.cc:256] RPC call timeout handler was delayed by -2.9319034090001423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 12:10:20.132822 10553 connection.cc:256] RPC call timeout handler was delayed by -2.9015208250000342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
I20260709 12:10:20.126754 12307 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.133577 12307 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:20.127472 12335 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.135429 12307 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.103199 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.137246 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.138437 10583 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:20.118549 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.139274 11930 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:20.140152 11930 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:20.141782 12307 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.142293 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.144604 12074 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:20.145885 10587 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:20.133119 10553 connection.cc:256] RPC call timeout handler was delayed by -2.8899645310000324s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
I20260709 12:10:20.128481 12243 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:20.147240 10553 connection.cc:256] RPC call timeout handler was delayed by -2.8854645310000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 12:10:20.147729 10553 connection.cc:256] RPC call timeout handler was delayed by -2.8643450950000897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4293s for a response
W20260709 12:10:20.148090 10553 connection.cc:256] RPC call timeout handler was delayed by -2.1437603350002519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 12:10:20.148444 10553 connection.cc:256] RPC call timeout handler was delayed by -2.1122603350002009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 12:10:20.148811 10553 connection.cc:256] RPC call timeout handler was delayed by -2.1077603350001937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 12:10:20.149185 10553 connection.cc:256] RPC call timeout handler was delayed by -2.070860335000134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 12:10:20.149468 10553 connection.cc:256] RPC call timeout handler was delayed by -1.9043603350000922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 12:10:20.149768 10553 connection.cc:256] RPC call timeout handler was delayed by -1.8728603350002686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 12:10:20.150063 10553 connection.cc:256] RPC call timeout handler was delayed by -1.8494603350002308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 12:10:20.150365 10553 connection.cc:256] RPC call timeout handler was delayed by -1.8422603350002191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
W20260709 12:10:20.150655 10553 connection.cc:256] RPC call timeout handler was delayed by -1.7873603350001304s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 12:10:20.151031 10553 connection.cc:256] RPC call timeout handler was delayed by -1.6640603350001584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20450000000000002s for a response
I20260709 12:10:20.102437 10437 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:20.151407 10553 connection.cc:256] RPC call timeout handler was delayed by -1.4939603350001107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
I20260709 12:10:20.150738 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:20.172726 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 228756us
I20260709 12:10:20.178670 10585 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:20.179996 10580 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.134630 12335 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:20.184455 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:20.184818 12074 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:10:20.191203 12074 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:20.191563 12337 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.192962 12213 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.191643 10586 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:20.184480 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.196060 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.152729 12267 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:20.196785 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:20.199465 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.214759 10578 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:20.218202 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 67606.7us
I20260709 12:10:20.219575 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.192611 12135 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.197130 11975 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:20.191689 12074 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:20.229341 12074 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.231562 12243 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.233510 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.233821 12335 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:20.234949 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.235718 10587 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:20.235831 12335 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.237813 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.238795 10582 raft_consensus.cc:2410] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:20.200815 12213 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:20.239176 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.197214 12274 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:20.224350 10583 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:10:20.226349 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.225485 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.200138 11906 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:20.213493 12171 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.002s
I20260709 12:10:20.207617 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:20.199512 12241 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:10:20.252918 11218 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Advancing to term 3
I20260709 12:10:20.257792 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.141867 11930 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.259441 10578 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:20.259063 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51703.2us
I20260709 12:10:20.260288 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.247457 11212 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.264168 10580 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:20.243093 12268 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:20.273291 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.223692 12344 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.274278 10584 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:20.272650 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:20.281637 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:20.282783 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57419.6us
I20260709 12:10:20.283994 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.285454 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.286999 10579 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:20.287122 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.287861 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:20.288385 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.260390 12213 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:20.291715 12319 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:20.292201 12213 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.292627 12319 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.293004 12319 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:20.257897 11906 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.293841 12319 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.294893 11906 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.267980 12074 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:20.295392 12319 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:20.303663 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 103881us
I20260709 12:10:20.263731 12243 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.245748 11975 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.308179 11218 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:20.273797 12267 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.260439 12337 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:20.312425 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.260372 12171 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.278088 12299 raft_consensus.cc:2764] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:20.273284 12268 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:20.316273 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.314190 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.304351 12057 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.315618 10580 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:20.316087 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.305135 12344 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:20.259145 12274 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.324272 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.324376 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:20.327194 10436 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.327442 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:20.314749 11888 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:20.311415 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.329291 11212 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.330169 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.330806 12339 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:20.331067 12267 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:20.322914 12243 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:20.332649 12243 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.334695 11975 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.339226 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:20.339529 11975 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:20.339846 10432 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.342306 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.343410 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.343667 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.344262 12344 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.345311 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.345844 12347 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:20.346019 10578 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:20.346395 12347 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:20.346830 12347 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.348193 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.349668 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.351378 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.351893 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:20.335341 10459 debug-util.cc:398] Leaking SignalData structure 0x7b0800308ea0 after lost signal to thread 12060
W20260709 12:10:20.353242 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad780 after lost signal to thread 12075
W20260709 12:10:20.353590 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3dc0 after lost signal to thread 12086
I20260709 12:10:20.353520 12299 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:20.354179 10459 debug-util.cc:398] Leaking SignalData structure 0x7b080028d120 after lost signal to thread 12119
I20260709 12:10:20.354442 12299 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:20.307103 12265 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.354908 12299 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:20.355475 10434 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.356581 12265 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.326778 12057 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:20.354575 10459 debug-util.cc:398] Leaking SignalData structure 0x7b0800340660 after lost signal to thread 12127
W20260709 12:10:20.359248 10459 debug-util.cc:398] Leaking SignalData structure 0x7b080034c920 after lost signal to thread 12136
I20260709 12:10:20.358744 12057 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:20.360255 10459 debug-util.cc:398] Leaking SignalData structure 0x7b08001d0ca0 after lost signal to thread 12167
I20260709 12:10:20.354110 12333 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:20.364383 12068 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.354205 12274 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.344767 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.318362 12268 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:20.334798 12345 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:10:20.375149 10436 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.316916 11906 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 12:10:20.393278 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.396816 12171 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.338858 12349 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.400697 12068 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:20.401328 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.350826 12276 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.402835 10587 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.410147 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:20.421164 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:20.422273 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72741.8us
W20260709 12:10:20.431357 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.421218 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.433094 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:20.390570 12292 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:20.427691 10432 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.396883 11907 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.452913 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:20.409106 12276 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:20.449991 12241 raft_consensus.cc:1064] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:20.467001 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:20.465898 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:20.450358 12263 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:20.472024 10432 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.470618 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:20.452000 12135 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:20.476125 12235 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.478330 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:20.486490 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.453035 12267 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:20.463671 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:20.469292 12241 raft_consensus.cc:1068] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:10:20.498737 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 171385us
I20260709 12:10:20.474462 12263 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:20.478940 11888 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:20.462798 11950 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 12:10:20.514935 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.492372 12228 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.521517 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.499828 12235 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:20.533334 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55685.8us
I20260709 12:10:20.508806 12263 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:20.547924 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77408.4us
I20260709 12:10:20.548976 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.552349 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.509220 12319 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:10:20.451556 12337 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.516021 12276 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:10:20.557145 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.540884 11950 consensus_queue.cc:1243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:10:20.538647 12228 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:20.569088 12057 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.554519 12307 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.556424 12328 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:10:20.576527 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.577201 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:20.577520 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56143.7us
I20260709 12:10:20.544983 11888 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.578343 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.580559 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.545854 12355 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:10:20.582901 10584 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.590155 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.558732 12068 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:20.597484 12228 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.576110 12213 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.607149 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.613236 10585 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:20.573597 12220 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:10:20.590854 12357 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:20.609906 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.619204 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.620075 10580 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:20.621165 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.622363 10587 raft_consensus.cc:2410] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:20.611186 12068 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.626536 10580 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:20.605684 12335 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:20.634610 12351 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:10:20.631314 12213 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:20.643791 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 86343.7us
W20260709 12:10:20.646626 12360 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:10:20.647567 10583 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:20.648411 12213 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.651115 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:20.651969 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:20.652295 10513 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.573884 12267 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.594158 12235 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.653077 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:20.661106 10507 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.663573 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:20.743408 10507 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:20.663918 10580 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.701515 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:20.644589 12356 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:20.591169 12354 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.729310 10581 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:20.742362 10429 raft_consensus.cc:606] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:20.825929 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1811) took 1200 ms. Trace:
I20260709 12:10:20.742465 12274 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:20.743793 10587 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 1.
I20260709 12:10:20.854276 10504 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:20.700459 12299 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.836213 10507 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.736320 10512 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:20.861632 10478 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.993s (SENT)
I20260709 12:10:20.865015 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:20.874944 10510 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:20.859582 10511 rpcz_store.cc:276] 0709 12:10:19.541840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.551160 (+  9320us) service_pool.cc:224] Handling call
0709 12:10:19.551510 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:19.582425 (+ 30915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:19.582459 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:19.591526 (+  9067us) log.cc:844] Serialized 26 byte log entry
0709 12:10:19.680092 (+ 88566us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:19.680154 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.680341 (+   187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:20.740850 (+1060509us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b2400167558. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa30fb0 00007f2c926a7488 00007f2c926a7be8 00007f2c9267f398 00007f2c9267a76a 00007f2c92679bb6
0709 12:10:20.741636 (+   786us) raft_consensus.cc:1685] finished
0709 12:10:20.741670 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:20.741958 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50844,"raft.queue_time_us":35526,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"spinlock_wait_cycles":21983104,"thread_start_us":88037,"threads_started":1,"wal-append.queue_time_us":162311}
I20260709 12:10:20.850299 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.871729 11888 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:20.884495 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.732666 12347 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.822052 12335 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:10:20.888378 11888 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.859357 10429 raft_consensus.cc:614] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:20.890477 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 334890us
I20260709 12:10:20.827862 12268 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.895282 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.901295 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.872016 12355 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:20.904156 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.906508 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:20.907689 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:20.907930 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46522.3us
W20260709 12:10:20.908800 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.858911 12317 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
W20260709 12:10:20.902460 12366 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:10:20.914671 10432 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:20.921790 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.904188 12346 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:20.875386 12274 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:20.956946 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.927723 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:20.960595 10508 raft_consensus.cc:2393] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:20.961493 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:20.911900 12228 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.882405 12356 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:20.968803 10506 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:10:20.966470 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50297.5us
I20260709 12:10:20.918027 12328 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:21.001449 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:20.927006 12335 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:20.964649 11216 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 LEARNER]: Advancing to term 4
I20260709 12:10:20.918828 12333 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:10:21.041188 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.012686 12328 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:21.044188 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:21.045272 10429 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:21.046016 10429 raft_consensus.cc:614] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:21.045332 12328 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:21.046885 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 163848us
I20260709 12:10:21.047957 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:21.049341 12333 consensus_queue.cc:1243] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:21.050603 12333 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.913170 12355 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.052091  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:10:21.060333 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 98932.1us
I20260709 12:10:21.083209 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.050977 10553 leader_election.cc:400] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 4.
I20260709 12:10:21.088981 10553 leader_election.cc:403] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:21.062561 10580 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:21.086607  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:10:21.091244 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 173265us
I20260709 12:10:21.092160 10552 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:21.091367 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.095258 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:21.052089 12372 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:10:21.051115 12333 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:21.053736 12328 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.061095 12102 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.103554 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.068486 12268 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.119510 12333 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.121079 12268 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:21.121646 12333 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.062654 12383 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.122059 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:21.108929 12382 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:21.123515 11888 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.123749 12382 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:21.113013 12367 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:10:21.113479 12274 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:21.114729 12370 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:10:21.115446 12102 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.117120 12355 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.118959  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:21.120239 12368 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:10:21.097286 11907 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.008s
W20260709 12:10:21.161062 12380 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:10:21.155196 12384 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:21.160537 12263 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.610s	user 0.005s	sys 0.010s
I20260709 12:10:21.156528 12388 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 4.
I20260709 12:10:21.121980 12268 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:21.191229 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 140449us
I20260709 12:10:21.193670 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:21.124261 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:21.188063 12384 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.163246 12347 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.138898 12299 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:21.209667 10513 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.200701 12384 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.225490 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:21.191157 12382 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:21.195545 12263 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:21.230019 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 134854us
I20260709 12:10:21.235776 12347 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:21.236795 12389 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:21.237485 12347 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.238093 12347 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:21.239456 11907 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:21.241298 12268 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.235811 12386 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.243048 12386 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:21.245958 12299 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:21.266706 12299 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:21.267419 12299 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.272429 12102 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.273478 12386 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:21.283248 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 89718.7us
I20260709 12:10:21.262246 12389 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:10:21.290110 12391 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.291052 12391 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.295265 12263 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.293033 12389 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:21.296168 12263 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:21.296829 12389 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.296916 12263 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.307960 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.320432 12068 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:21.327406 12365 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.702s	user 0.000s	sys 0.002s
I20260709 12:10:21.319067 11216 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:21.336395 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47159.4us
I20260709 12:10:21.337961 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.340440 12223 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:21.340895 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.341856 12223 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:21.342486 12223 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:21.343183 12223 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.343757 10509 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:21.306691 12391 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.343253 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.344869 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:21.329548 12263 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.314195 12333 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:21.349624 10508 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:21.339787 12224 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:10:21.354660 12343 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:21.357779 12346 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:21.358314 12346 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:21.365383 12389 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.366480 10478 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.369647 12389 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.371914 10508 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.368849 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:21.373473 12346 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.377377 12403 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:21.388922 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44151.5us
I20260709 12:10:21.384258 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:21.398782 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:21.397967 12377 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.227s	user 0.000s	sys 0.002s
I20260709 12:10:21.401438 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.406435 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.406733 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:21.408766 10510 raft_consensus.cc:2410] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:21.397071 12333 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:21.398207 12389 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:21.411764 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.402539 12346 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.421481 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.423197 12342 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:21.423663 12342 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:21.424057 12342 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.424122 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:21.425329 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 289) took 1017 ms. Trace:
I20260709 12:10:21.425753 11215 rpcz_store.cc:276] 0709 12:10:20.408383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.419029 (+ 10646us) service_pool.cc:224] Handling call
0709 12:10:21.423879 (+1004850us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:21.425232 (+  1353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5801216}
I20260709 12:10:21.427939 12384 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.411159 12343 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.411824 12333 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:21.430085 12333 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:21.431267 10580 raft_consensus.cc:2410] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:21.431615 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.420076 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:21.422130 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:21.433321 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:21.441679 12386 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:21.445885 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:21.446578 10587 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.455983 12342 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.463351 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:21.467466 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.468286 10578 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:21.469485 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.475600 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.476684 10582 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:21.423677 12398 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.479825 12384 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:21.438752 12343 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.481097 12343 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.481783 12343 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.430528 12223 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:21.486510 12394 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.248s	user 0.002s	sys 0.002s
I20260709 12:10:21.487614 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 296) took 1684 ms. Trace:
I20260709 12:10:21.432237 12220 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:21.488298 12373 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.429s	user 0.002s	sys 0.000s
I20260709 12:10:21.493695 12223 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.496097 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1473) took 1261 ms. Trace:
I20260709 12:10:21.496878 12347 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.498306 10578 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:21.456707 10587 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:21.516207 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.518625 10582 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:21.524943 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:21.514168 10584 rpcz_store.cc:276] 0709 12:10:20.234156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.278278 (+ 44122us) service_pool.cc:224] Handling call
0709 12:10:21.495852 (+1217574us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400334854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:21.496000 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678303104}
I20260709 12:10:21.534847 12347 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.507769 11217 rpcz_store.cc:276] 0709 12:10:19.803367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.803576 (+   209us) service_pool.cc:224] Handling call
0709 12:10:19.804075 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:20.112530 (+308455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.458284 (+1345754us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400591854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:21.487421 (+ 29137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327400320}
I20260709 12:10:21.544255 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:21.540884 12299 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.546166 12276 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:21.547235 12267 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:10:21.548053 12267 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:21.557543 12356 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.559269 12299 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.563779 12382 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.583230 12319 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:21.583913 12319 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:21.585453 12319 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.592973 12319 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:21.596455 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 173515us
I20260709 12:10:21.597417 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.600265 12299 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.577257 11907 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:21.603354 11907 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.606438 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.610697 12384 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:21.606585 11907 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.635408 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:21.642948 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:21.647667 10578 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.644557 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:21.648250 10578 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:21.666296 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:21.667570 12344 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.668094 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:21.668242 12344 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:21.669145 12344 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.669523 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:21.670603 12263 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:21.671501 12263 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:21.673255 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.673061 12263 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.674363 12346 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.675390 12077 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.676237 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.655526 12223 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:21.675990 12077 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.681011 12193 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:21.681959 12193 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.646325 12383 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.715963 10580 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.716610 10580 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:21.717614 10580 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.719462 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:21.720021 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 1846) took 1057 ms. Trace:
W20260709 12:10:21.720618 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44502.9us
W20260709 12:10:21.683995 12392 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.441s	user 0.004s	sys 0.000s
I20260709 12:10:21.720633 10581 rpcz_store.cc:276] 0709 12:10:20.662882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.109645 (+446763us) service_pool.cc:224] Handling call
0709 12:10:21.155114 (+ 45469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:21.209212 (+ 54098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:21.209271 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:21.210090 (+   819us) log.cc:844] Serialized 26 byte log entry
0709 12:10:21.523233 (+313143us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b18003ed1c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:21.666709 (+143476us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:21.666786 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.666967 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:21.719511 (+ 52544us) raft_consensus.cc:1685] finished
0709 12:10:21.719546 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.719919 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55333,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":10222,"spinlock_wait_cycles":6239616,"thread_start_us":196648,"threads_started":2,"wal-append.queue_time_us":487631}
I20260709 12:10:21.645081 12382 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:21.682847 12077 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.687085 12223 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:21.727641 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:21.728621 10433 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:21.677477 12335 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:21.679473 11907 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.727380 10580 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.732786 10437 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:21.741397 10432 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:21.742455 10583 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:21.747225 10583 pending_rounds.cc:85] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 1
I20260709 12:10:21.747363 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:21.747637 10583 pending_rounds.cc:107] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:21.748045 10583 raft_consensus.cc:2889] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:21.749703 12342 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:21.757223 12375 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.890s	user 0.000s	sys 0.003s
I20260709 12:10:21.761605 10578 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.765264 12344 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.770088 10507 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.772737 10583 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:21.781096 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.792126 10432 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:21.796713 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:21.798944 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:21.799814 10430 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.808772 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:21.707753 12328 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:21.810177 10580 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:21.811101 10580 pending_rounds.cc:85] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 1
I20260709 12:10:21.700326 12356 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:21.811456 10580 pending_rounds.cc:107] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:21.811771 10580 raft_consensus.cc:2889] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:21.698385 12387 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:21.664243 12343 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.742735 12223 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.860548 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:21.861837 10432 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:21.762915 10578 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.856633 10435 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:21.867034 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:21.877554 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:21.878618 10509 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.861668 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:21.707499 12383 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.841771 12356 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.892781 12193 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.875546 12387 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.892848 12343 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:21.895226 12387 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:21.786224 12409 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 12:10:21.836619 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 101193us
W20260709 12:10:21.905985 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.734421 12267 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.745935 11907 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.810236 10582 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.810935 12344 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:21.912662 12403 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.514s	user 0.009s	sys 0.006s
I20260709 12:10:21.912923 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 233) took 9839 ms. Trace:
I20260709 12:10:21.913626 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:21.916082 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:21.816664 12349 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:21.809576 12295 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.810391 12057 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.923168 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:21.821528 12398 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:21.901665 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:21.923316 11907 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.745136 12068 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.155s
I20260709 12:10:21.984712 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:21.984969 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:21.986254 10432 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:21.947190 10507 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:21.914407 12241 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.903923 12356 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:21.953588 12403 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:21.979419 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72609.5us
I20260709 12:10:21.981077 10430 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:21.922308 10582 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:21.987720 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 109) took 1555 ms. Trace:
I20260709 12:10:21.998198 10507 rpcz_store.cc:276] 0709 12:10:20.432655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.432801 (+   146us) service_pool.cc:224] Handling call
0709 12:10:20.454655 (+ 21854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:20.660965 (+206310us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b540014d820. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae11c0 00007f2c92ae2231 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c
0709 12:10:21.769552 (+1108587us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b58002af5d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a3b0aa 00007f2c92abd447 00007f2c92ad5fe7 00007f2c92ad77de 00007f2c92ae1b49 00007f2c92ae2416 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4
0709 12:10:21.987338 (+217786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.987649 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1578616576}
I20260709 12:10:21.929519 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:22.002297 10513 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:10:22.002781 10513 pending_rounds.cc:85] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 5
I20260709 12:10:22.003043 10513 pending_rounds.cc:107] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:10:22.003377 10513 raft_consensus.cc:2889] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:21.976497 12299 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.975416 12349 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:21.984172 11212 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.933669 10336 rpcz_store.cc:276] 0709 12:10:12.073450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.073621 (+   171us) service_pool.cc:224] Handling call
0709 12:10:21.912831 (+9839210us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:19.920634 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:19.995350 (+ 74716us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:19.995454 (+   104us) write_op.cc:435] Acquired schema lock
0709 12:10:19.995485 (+    31us) tablet.cc:662] Decoding operations
0709 12:10:20.103004 (+107519us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:20.145792 (+ 42788us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:20.145843 (+    51us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:10:20.146914 (+  1071us) tablet.cc:701] Row locks acquired
0709 12:10:20.147008 (+    94us) write_op.cc:260] PREPARE: finished
0709 12:10:20.147370 (+   362us) write_op.cc:270] Start()
0709 12:10:20.147709 (+   339us) write_op.cc:276] Timestamp: P: 1783599020147271 usec, L: 0
0709 12:10:20.147755 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:10:20.199414 (+ 51659us) log.cc:844] Serialized 1672 byte log entry
0709 12:10:20.585135 (+385721us) op_driver.cc:464] REPLICATION: finished
0709 12:10:20.908082 (+322947us) write_op.cc:301] APPLY: starting
0709 12:10:20.915463 (+  7381us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:20.976317 (+ 60854us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:21.062302 (+ 85985us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:21.116979 (+ 54677us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:21.117408 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:10:21.117489 (+    81us) write_op.cc:312] APPLY: finished
0709 12:10:21.118040 (+   551us) log.cc:844] Serialized 41 byte log entry
0709 12:10:21.184669 (+ 66629us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:21.188530 (+  3861us) write_op.cc:454] Released schema lock
0709 12:10:21.190539 (+  2009us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:21.212315 (+ 21776us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:21.214330 (+  2015us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2845530,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":296659,"apply.run_cpu_time_us":5662,"apply.run_wall_time_us":318049,"num_ops":4,"prepare.queue_time_us":143159,"prepare.run_cpu_time_us":14317,"prepare.run_wall_time_us":458067,"raft.queue_time_us":145967,"raft.run_cpu_time_us":17501,"raft.run_wall_time_us":276602,"replication_time_us":413369,"thread_start_us":389700,"threads_started":4,"wal-append.queue_time_us":114560}]]}
I20260709 12:10:21.990507 12074 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.914268 12333 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:21.923931 12267 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:22.015218 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:21.999292 12139 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:10:21.980114 12398 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:22.018174 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 290592us
I20260709 12:10:22.019213 10506 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.993054 12387 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.976198 12295 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:21.978740 12057 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:21.995493 12241 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:22.051164 10586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.093s	sys 0.125s
I20260709 12:10:22.041890 10578 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
W20260709 12:10:22.054654 10586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.094s	sys 0.125s
I20260709 12:10:22.054702 10578 pending_rounds.cc:85] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 1
I20260709 12:10:22.055428 10578 pending_rounds.cc:107] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:22.055706 10578 raft_consensus.cc:2889] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:22.085938 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44497.6us
I20260709 12:10:22.017436 11950 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:10:22.096940 10578 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:22.040719 12398 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.106007 11219 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.099917 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.104681 12171 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:10:22.111466 12171 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:22.112007 12171 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:22.111920 10333 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 3, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:22.111742 12344 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:22.113061 12171 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.114843 12171 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.115422 11950 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:22.115918 11950 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:22.108047 12385 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:10:22.116905 11950 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.118219 11950 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.122520 12241 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.124334 12362 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:22.124857 12057 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.124362 12295 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:22.051679 12396 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 12:10:22.127539 12267 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:22.128012 12220 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.005s
I20260709 12:10:22.128293 12267 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.128554 12220 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.128844 12267 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.129341 12220 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:10:22.111595 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.132889 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1466) took 1901 ms. Trace:
I20260709 12:10:22.133299 10586 rpcz_store.cc:276] 0709 12:10:20.231776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.247850 (+ 16074us) service_pool.cc:224] Handling call
0709 12:10:20.272430 (+ 24580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:20.280661 (+  8231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:20.280701 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:21.205916 (+925215us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b18001b66a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a
0709 12:10:21.531318 (+325402us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:10:21.532079 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 12:10:22.055568 (+523489us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:22.055662 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.055782 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:22.132627 (+ 76845us) raft_consensus.cc:1685] finished
0709 12:10:22.132651 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:22.132815 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3251,"raft.queue_time_us":10050,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151,"spinlock_wait_cycles":23668352,"thread_start_us":522895,"threads_started":1,"wal-append.queue_time_us":488684}
I20260709 12:10:22.135581 12333 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:22.136344 10579 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:10:22.107002 10582 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.027630 12074 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.137151 10582 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.140219 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:22.141455 12388 consensus_queue.cc:1175] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" responder_term: 4 status { last_received { term: 4 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:10:22.146884 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:22.165073 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:22.168748 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.169788 10430 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:22.171746 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.173022 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:22.180968 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.186535 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:22.189514 12396 consensus_queue.cc:1243] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:22.194051 11212 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:22.195818 10436 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:22.197111 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:22.197602 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 454644us
I20260709 12:10:22.051600 12403 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 12:10:22.199362 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:22.202610 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 103932us
I20260709 12:10:22.090023 12135 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 12:10:22.051875 12387 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.179821 12057 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.220142 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.221206 10436 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:22.221553 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.184727 12383 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.184592 12068 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:22.201174 12220 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:22.206954 12333 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:22.209301 12400 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:22.205567 12074 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.263149 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.266026 10512 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:22.267369 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.204339 12292 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:10:22.289052 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.304800 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:22.276889 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.228421 12224 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:22.304085 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.303373 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:22.314911 10436 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:22.252707 12220 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:22.248504 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:22.319348 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 281243us
I20260709 12:10:22.321112 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.257587 12333 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:22.220964 12223 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:22.314389 10435 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:22.332190 10513 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:22.251164 12295 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:10:22.251744 12213 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:10:22.257148 12068 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.313231 12268 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:22.309126 12400 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:22.204980 12349 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.366219 10509 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:22.320001 12220 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.350394 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:22.372192 10580 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.351142 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:22.308887 12183 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:10:22.368273 10513 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:22.319087 12224 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:22.341849 12267 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.342741 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.380988 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:22.381996 10509 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:22.379058 11218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 298) took 2060 ms. Trace:
I20260709 12:10:22.340127 12337 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 12:10:22.386488 10429 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.005s
I20260709 12:10:22.390489 10429 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:22.383342 10583 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:22.384133 10553 connection.cc:256] RPC call timeout handler was delayed by -1.03248646499992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 12:10:22.340451 12413 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:10:22.384578 11218 rpcz_store.cc:276] 0709 12:10:20.318857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.319055 (+   198us) service_pool.cc:224] Handling call
0709 12:10:21.691168 (+1372113us) spinlock_profiling.cc:243] Waited 8.78 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:21.691451 (+   283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:21.710141 (+ 18690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:21.710195 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:21.711185 (+   990us) log.cc:844] Serialized 52 byte log entry
0709 12:10:22.020827 (+309642us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:22.020878 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.020966 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:22.378709 (+357743us) raft_consensus.cc:1685] finished
0709 12:10:22.378730 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:22.378946 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30478,"raft.queue_time_us":19449,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":63543,"spinlock_wait_cycles":20191488,"thread_start_us":176555,"threads_started":1,"wal-append.queue_time_us":505536}
I20260709 12:10:22.380276 12224 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:22.406647 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0234864649999054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 12:10:22.412982 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:22.416735 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.419891 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:22.423348 10509 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:22.351761 12415 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:10:22.425855 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:22.383255 12328 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:22.427060 12422 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:10:22.428072 10581 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:22.429629 12068 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:22.430891 10384 auto_rebalancer.cc:893] Removing replace marker failed: Replica 99d42e7288f145f1bd5af212fd062995 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:22.431532 12384 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.435436 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.435559 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:22.440400 11219 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:22.441095 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.441953 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:22.485345 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43478.3us
I20260709 12:10:22.443607 10578 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:22.485817 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.449390 10580 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:22.486793 10586 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:22.425755 12135 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:22.420176 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.490443 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.490516 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.491418 10581 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:22.491832 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.372836 12268 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:22.466832 10384 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:10:22.474118 12077 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.005s
W20260709 12:10:22.467374 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 172698us
I20260709 12:10:22.467890 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:22.468852 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 196903us
I20260709 12:10:22.474494 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.498898 10430 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:22.499969 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.500706 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.500836 10430 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:22.501562 10578 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:22.502902 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:22.481032 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.492614 10429 raft_consensus.cc:2368] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:22.507701 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.508936 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.509954 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.493186 10583 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:22.511261 10433 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:22.383040 12333 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.515861 10507 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:22.495522 10384 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
W20260709 12:10:22.516865 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.517843 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.517976 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.530814 10437 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:22.518874 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.535197 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.474179 12428 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:22.541016 12428 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:22.492564 10582 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:22.538481 10429 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:22.539721 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:22.543735 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:22.498262 12343 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.496654 12387 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.467929 12328 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.539902 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.581966 10437 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:22.541520 12428 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.588186 10431 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.589280 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:22.549213 10582 pending_rounds.cc:85] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 1
I20260709 12:10:22.585250 11215 raft_consensus.cc:2410] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:22.580724 12263 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:22.596536 12426 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:10:22.609772 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.597205 12433 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.598510 12432 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:10:22.604069 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.600415 10581 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:22.501829 12077 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.636794 12328 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:22.637334 12077 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.582852 12347 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:22.638286 12347 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:22.639389 12268 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.639348 12347 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.512415 12288 consensus_queue.cc:1059] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 12:10:22.592410 12422 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:10:22.610044  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:22.647425 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1501) took 1204 ms. Trace:
I20260709 12:10:22.647846 10585 rpcz_store.cc:276] 0709 12:10:21.443198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.443470 (+   272us) service_pool.cc:224] Handling call
0709 12:10:21.443897 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:21.444363 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:21.444400 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:21.785686 (+341286us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:10:21.820021 (+ 34335us) log.cc:844] Serialized 26 byte log entry
0709 12:10:22.035121 (+215100us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:22.083608 (+ 48487us) log.cc:844] Serialized 14 byte log entry
0709 12:10:22.379339 (+295731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.379427 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:22.647054 (+267627us) raft_consensus.cc:1685] finished
0709 12:10:22.647179 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:22.647348 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8046,"prepare.queue_time_us":52815,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":116394,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":251,"spinlock_wait_cycles":33957760,"thread_start_us":431043,"threads_started":2,"wal-append.queue_time_us":434105}
I20260709 12:10:22.609236 12263 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:22.650018 12263 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.583279 12431 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:22.619990 10437 raft_consensus.cc:2393] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:22.601995 10582 pending_rounds.cc:107] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:22.615442 12299 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:22.626839 10429 pending_rounds.cc:85] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:22.675660 10429 pending_rounds.cc:107] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:22.675989 10429 raft_consensus.cc:2889] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:22.538853 12074 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:10:22.580586 12057 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:10:22.632862 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.580979 12383 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:10:22.629827 11950 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.615680 12223 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.669693 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:22.694903 11950 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.697325 11950 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:22.533535 12400 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.697876 11950 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:22.698222 11950 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:22.701424 10437 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:22.702407 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:22.583784 12323 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.641980 12288 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:22.708083 12288 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:22.709092 12288 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.710415 12288 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.674819 10582 raft_consensus.cc:2889] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:22.711778 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1513) took 1076 ms. Trace:
I20260709 12:10:22.712141 10582 rpcz_store.cc:276] 0709 12:10:21.634964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.646601 (+ 11637us) service_pool.cc:224] Handling call
0709 12:10:21.710571 (+ 63970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:22.711440 (+1000869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.711703 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:22.683368 12445 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.689014 10579 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:22.703536 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:22.704371 10437 raft_consensus.cc:2368] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:22.728713 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:22.754568 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 255254us
I20260709 12:10:22.756889 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:22.772372 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68916.9us
I20260709 12:10:22.775748 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:22.781472 10438 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:22.788157 10432 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:22.687593 12299 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.750000 12445 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.815191 12424 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:22.831125 12398 consensus_meta.cc:287] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.004s
W20260709 12:10:22.831949 12407 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.900s	user 0.002s	sys 0.004s
I20260709 12:10:22.831799 12398 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.832552 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:22.833488 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 117) took 1121 ms. Trace:
I20260709 12:10:22.803475 12068 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:10:22.819527 12439 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:10:22.752526 12323 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:22.795650 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.780575 12319 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:22.814281 12440 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:22.763028 11950 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.797421 12421 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:10:22.828207 11907 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.835s	user 0.012s	sys 0.002s
I20260709 12:10:22.834792 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:22.823849 12241 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:22.852489 12241 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:22.853601 12241 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.856902 12323 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.857342 12337 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:22.857931 12337 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:22.754329 12432 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:22.830787 12424 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:22.839272 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:22.842192 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 297264us
I20260709 12:10:22.842715 10431 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:22.849741 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:22.880623 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.886323 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:22.890798 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:22.900375 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:22.840292 12415 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:22.905804 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56200.9us
I20260709 12:10:22.878830 12241 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.908464 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:22.885051 12425 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.892745 12424 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:22.848308 12319 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.919317 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:22.939579 12299 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:22.907781 12263 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.941322 12223 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:22.943536 12445 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:22.900767 12391 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:22.961597 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:22.908795 12398 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:22.908392 12068 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.884254 11907 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.910246 12387 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:22.974100 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.023926 10580 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.965168 10585 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:10:22.986593 10583 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.889262 12432 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.833976 10434 rpcz_store.cc:276] 0709 12:10:21.711735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.722006 (+ 10271us) service_pool.cc:224] Handling call
0709 12:10:21.884868 (+162862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.895356 (+ 10488us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:22.261555 (+366199us) log.cc:844] Serialized 14 byte log entry
0709 12:10:22.538148 (+276593us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:22.549064 (+ 10916us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:22.583038 (+ 33974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.833170 (+250132us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b54003a98d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:22.833233 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:22.833411 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":220163,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":13632,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3774,"spinlock_wait_cycles":23309568,"thread_start_us":544283,"threads_started":2,"wal-append.queue_time_us":366471}
I20260709 12:10:22.914822 12415 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:22.917148 12421 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.984016 12391 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:22.968036 12057 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:23.065587 12057 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.066061 12057 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.067155 12415 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.069373 12391 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.899072 12428 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.042774 11907 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:22.967803 12452 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:22.980770 12446 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.153s	user 0.000s	sys 0.002s
W20260709 12:10:23.005151 12220 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.622s	user 0.005s	sys 0.008s
I20260709 12:10:22.967469 11950 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.971729 12268 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.971841 12362 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.023455 12387 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.107125 12421 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:23.046105 12224 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.587s	user 0.006s	sys 0.006s
I20260709 12:10:22.969812 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:22.984903 12077 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.159s
W20260709 12:10:23.359903 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:23.174593 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:23.147796 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.022s	sys 0.053s
I20260709 12:10:23.357038 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1551) took 1210 ms. Trace:
W20260709 12:10:23.374168 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.027s	sys 0.062s
W20260709 12:10:23.301916 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 393528us
I20260709 12:10:23.046854 12223 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:23.377966 10580 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:23.379428 10585 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:10:23.379554 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.379889 11215 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:23.133487 10583 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:23.377017 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.382723 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:23.374933 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:23.295321 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.375289 10436 rpcz_store.cc:276] 0709 12:10:22.146127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.324755 (+178628us) service_pool.cc:224] Handling call
0709 12:10:22.345340 (+ 20585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:22.351775 (+  6435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:22.351816 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:22.369125 (+ 17309us) log.cc:844] Serialized 26 byte log entry
0709 12:10:22.780352 (+411227us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:22.780416 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.780518 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:23.356704 (+576186us) raft_consensus.cc:1685] finished
0709 12:10:23.356729 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.356946 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5958,"raft.queue_time_us":42115,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":63116,"spinlock_wait_cycles":1858944,"thread_start_us":385722,"threads_started":1,"wal-append.queue_time_us":601263}
W20260709 12:10:23.340399 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.376132 12241 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:23.373622 12387 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.397431 12439 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.147277 12346 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.398057 12439 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.398324 12346 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.399456 12391 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.400676 12439 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.401337 12068 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:23.401892 12391 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.391080 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:23.295426 12382 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.390183 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.403915 10585 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.230229 12299 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.392138 10583 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:23.393440 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.320330 12445 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.416390 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.408339 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:23.287325 12288 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.109038 12432 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:23.299023 12440 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.058859 12385 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.327s
I20260709 12:10:23.376575 12139 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:23.367143 12263 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.391724 12384 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.418249 10583 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.354851 12057 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.413705 12241 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.424358 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1526) took 1015 ms. Trace:
I20260709 12:10:23.428979 10584 rpcz_store.cc:276] 0709 12:10:22.408569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.408733 (+   164us) service_pool.cc:224] Handling call
0709 12:10:22.409199 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:22.715310 (+306111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:22.715348 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:22.715397 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:22.715436 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.424060 (+708624us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b540010c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:23.424132 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.424293 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703406976}
I20260709 12:10:23.429731 12213 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.429880 12241 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.430186 12213 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:23.424281 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:23.410524 12415 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.431257 12213 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.432452 12213 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.427600 12440 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.437225 12440 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.438071 12319 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.429891 12057 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.419613 12299 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.401916 12068 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.440169 10583 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:23.441210 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.441643 12068 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.442368 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.442524 12383 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:23.442286 12139 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.443610 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.444249 12421 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.444640 12139 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.423805 12337 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.445194 10583 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:23.419371 12382 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.447449 12382 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:23.417140 12342 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:23.421880 12445 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.450402 12445 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.430757 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.452589 10584 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.377609 12220 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.427750 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.386686 12268 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:23.375419 12224 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.425665 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.455027 10583 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:23.422024 12347 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:23.418226 12439 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.445312 12333 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.468546 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.469403 10436 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:23.471211 12445 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.463788 12268 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.473712 10584 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:23.473856 12268 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.464213 10585 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:23.451725 12383 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:23.459164 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.377597 12362 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:23.466300 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.377692 12452 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.466866 12342 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:23.454540 12337 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.457978 12384 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.490710 12337 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.491575 12171 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:23.492182 12171 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:23.492945 12337 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:23.493475 12171 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.467422 12346 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:23.425004 12428 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.492945 12384 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.499478 12328 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.499949 12328 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:23.500262 12464 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:10:23.500831 12464 raft_consensus.cc:1068] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:23.500905 12328 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.477602 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:23.476711 12391 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.503106 12391 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.479637 10508 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:23.476115 12383 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.489327 10510 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:23.489590 12342 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:23.463734 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.498327 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:23.532364 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.533390 10580 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:23.534340 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.426263 12288 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.496707 12171 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:23.529383 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51870us
I20260709 12:10:23.481611 12347 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:23.459702 12223 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.496466 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.546363 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.538681 12383 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:23.511962 12224 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.537385 12328 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.549682 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:23.538429 12213 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.539464 12403 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:23.545666 12346 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.551107 11221 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.537561 11907 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:23.544834 12384 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.569128 11907 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.569340 12384 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:23.545161 12396 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.571596 11907 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:23.572481 12450 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.616s	user 0.000s	sys 0.002s
I20260709 12:10:23.572865 12288 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.545607 12339 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 12:10:23.574154 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.575523 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:23.577162 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1531) took 1166 ms. Trace:
I20260709 12:10:23.578276 11950 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.579741 12057 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:23.579914 12241 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.580333 12057 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.580490 12241 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.581455 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:23.582643 12241 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.557821 12346 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:23.564803 12448 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.621s	user 0.003s	sys 0.000s
I20260709 12:10:23.566526 12391 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.567435 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:23.571322 12396 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.583148 12263 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:23.583428 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.641920 10437 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:23.584465 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:23.587371 10463 heartbeater.cc:646] Failed to heartbeat to 127.9.11.126:42677 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.11.126:42677 timed out after 15.000s (SENT)
I20260709 12:10:23.587674 10429 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:23.551903 12139 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:23.588086 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:23.588757 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:23.589406 12413 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:23.589596 12267 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:23.587674 10507 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.557915 12403 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:23.578862 11950 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.577566 10586 rpcz_store.cc:276] 0709 12:10:22.410913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.487898 (+ 76985us) service_pool.cc:224] Handling call
0709 12:10:22.488319 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:22.488952 (+   633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:22.488999 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:22.489671 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 12:10:22.721207 (+231536us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:22.721254 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.721342 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:23.498201 (+776859us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b48000b19d8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f 00007f2c97af6f37 00007f2c97b0b55c 00007f2c97b0b508 00007f2c97b0b488 00007f2c97b0b443 00007f2c97b0a6ec 00007f2c96a34a96
0709 12:10:23.571848 (+ 73647us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:23.576917 (+  5069us) raft_consensus.cc:1685] finished
0709 12:10:23.576942 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.577092 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41837,"raft.queue_time_us":174,"raft.run_cpu_time_us":3025,"raft.run_wall_time_us":1006128,"spinlock_wait_cycles":38133888,"thread_start_us":231120,"threads_started":1,"wal-append.queue_time_us":403598}
I20260709 12:10:23.693866 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.695006 10586 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:23.589533 10552 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 2.036s (SENT) [suppressed 71 similar messages]
I20260709 12:10:23.595569 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:23.615238 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1534) took 1203 ms. Trace:
I20260709 12:10:23.615661 12362 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:23.620633 12452 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.620952 12347 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.552111 10582 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.625222 10587 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:23.631577 10578 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:23.730425 10587 pending_rounds.cc:85] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 2
I20260709 12:10:23.730751 10587 pending_rounds.cc:107] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:23.582757 12057 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.612200 12213 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:23.694751 12267 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:23.739282 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.740200 10507 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:23.651510 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.655755 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:23.741425 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 85763.1us
I20260709 12:10:23.658859 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.625897 12299 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:23.654449 12413 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:23.654439 12463 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.689283 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1535) took 1193 ms. Trace:
I20260709 12:10:23.696426 11221 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:23.697686 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.749154 10512 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:23.695806 10433 raft_consensus.cc:2376] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4. Re-sending same reply.
I20260709 12:10:23.622120 12387 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.722641 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.731027 10587 raft_consensus.cc:2889] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:23.751331 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.748422 10581 rpcz_store.cc:276] 0709 12:10:22.496166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.665671 (+169505us) service_pool.cc:224] Handling call
0709 12:10:22.700000 (+ 34329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:22.750686 (+ 50686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:22.750747 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:22.751506 (+   759us) log.cc:844] Serialized 26 byte log entry
0709 12:10:23.505858 (+754352us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b18002d8080. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:23.546032 (+ 40174us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:23.546079 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.546227 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:23.688850 (+142623us) raft_consensus.cc:1685] finished
0709 12:10:23.688885 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.689204 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21840,"raft.queue_time_us":50917,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":25721,"spinlock_wait_cycles":6555904,"thread_start_us":39937,"threads_started":1,"wal-append.queue_time_us":900166}
I20260709 12:10:23.622026 12384 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:23.706665 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.754443 10586 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:23.737984 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.755962 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.745940 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.757642 10504 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:23.652580 12263 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.657413 12139 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.761039 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.734649 12220 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.756783 10578 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:23.763304 12267 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.763617 12413 consensus_queue.cc:1059] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:23.764012 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.764626 12413 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.765168 12413 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:23.765646 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.766053 12413 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.771701 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.772084 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 1828) took 1509 ms. Trace:
I20260709 12:10:23.772543 10510 rpcz_store.cc:276] 0709 12:10:22.262617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.262809 (+   192us) service_pool.cc:224] Handling call
0709 12:10:23.771848 (+1509039us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400228dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:23.772013 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2784034560}
I20260709 12:10:23.773247 12387 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.773860 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.774176 10505 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:23.754369 12463 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.777361 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.778261 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.779045 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.780484 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.780565 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.781463 10581 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:23.748615 12288 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:23.781994 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:23.784825 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.783771 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.787413 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:23.787885 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.789008 10436 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:23.788048 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.789327 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.730780 12421 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:10:23.790030 10437 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:23.790768 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.794915 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:23.791651 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:23.793733 10433 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:23.798766 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:23.755666 12068 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:23.800299 10581 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:23.804761 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.803052 10586 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:23.699944 10579 rpcz_store.cc:276] 0709 12:10:22.411904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.440688 (+ 28784us) service_pool.cc:224] Handling call
0709 12:10:23.614831 (+1174143us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400090954. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:23.615131 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272653696}
I20260709 12:10:23.781105 12425 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.806522 12385 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.806475 12413 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.764050 12267 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:10:23.814752 12396 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.805455 10510 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:23.783658 12319 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:10:23.818536 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.827105 10578 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:23.848457 10510 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:23.849757 10510 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.850163 10578 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:23.797626 12415 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:23.793979 12346 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:23.808159 12139 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.841127 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.845470 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:23.861331 10578 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:23.837349 12223 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.800232 12462 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:10:23.855756 12171 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.885087 12462 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.885259 11950 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.885573 12462 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:23.886713 12213 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:23.879570 12441 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:23.863945 12424 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.896s	user 0.015s	sys 0.000s
I20260709 12:10:23.784904 12403 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:23.895483 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 305970us
I20260709 12:10:23.864243 12415 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:23.882879 12268 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 12:10:23.902993 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:23.903591 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.883378 12224 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.906898 12337 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.909211 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:23.899577 12415 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:23.911118 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 121605us
I20260709 12:10:23.914367 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.901115 12424 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.917032 12424 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:23.918547 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:23.929173 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.885118 12171 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.842845 12385 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.912588 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:23.936513 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:23.943156 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.913101 12346 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:23.947602 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:23.951917 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:23.954857 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:23.936164 12477 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:10:23.910539 12224 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.886606 12462 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:23.932657 12424 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.958830 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:23.920799 12337 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:23.970109 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:23.943145 11214 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:23.951735 12346 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.953331 12478 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.979990 12346 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.981508 12268 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:23.983282 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 1826) took 1722 ms. Trace:
I20260709 12:10:23.987403 10513 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.990331 12319 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:23.994997 11215 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:10:24.042430 11215 pending_rounds.cc:85] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 4
I20260709 12:10:24.042845 11215 pending_rounds.cc:107] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:10:24.043435 11215 raft_consensus.cc:2889] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:23.996678 12224 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:23.959635 12267 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:23.995026 11219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 303) took 2190 ms. Trace:
I20260709 12:10:24.008065 12387 raft_consensus.cc:1064] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:24.045063 11219 rpcz_store.cc:276] 0709 12:10:21.804839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.805024 (+   185us) service_pool.cc:224] Handling call
0709 12:10:23.994819 (+2189795us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54004074d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:23.994938 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509384960}
I20260709 12:10:24.015424 10508 rpcz_store.cc:276] 0709 12:10:22.260823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.261029 (+   206us) service_pool.cc:224] Handling call
0709 12:10:23.983036 (+1722007us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540038f9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:23.983210 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2669184768}
I20260709 12:10:23.975848 12213 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.047482 10429 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
W20260709 12:10:24.029907 12482 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:10:24.049386 12171 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.049993 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.011348 12263 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:24.051477 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.052806 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.053437 10579 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:24.053968 10580 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:24.054208 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.006992 12463 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.055864 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.056469 10579 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.057370 12441 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:24.057993 12441 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.028472 12319 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:24.060359 12478 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:24.060022 12319 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.061646 12319 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.065680 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.066951 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.067683 10583 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:24.073309 12337 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.075704 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 273) took 1168 ms. Trace:
I20260709 12:10:24.076239 11214 rpcz_store.cc:276] 0709 12:10:22.907381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.917510 (+ 10129us) service_pool.cc:224] Handling call
0709 12:10:23.567215 (+649705us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:24.075442 (+508227us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b540051fed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:24.075584 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168879232}
I20260709 12:10:24.068363 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.081648 12385 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:24.083169 12383 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
W20260709 12:10:24.084906 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.087280 12464 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.096187 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.099045 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.099738 10437 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:24.102280 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.102945 10433 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.103950 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.105389 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.106021 10430 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.107260 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.108731 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.110092 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.089949 10583 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:24.111800 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.097687 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.113977 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.114269 10507 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:24.114749 10435 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:24.116287 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.116919 10583 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:24.123247 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:24.123277 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.124809 10436 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.124917 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.135747 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.086860 12441 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.141718 10435 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:24.178853 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.090598 10582 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:24.196382 10430 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:24.124821 12464 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:24.206176 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 83046.5us
I20260709 12:10:24.210692 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.210402 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.213471 10430 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:24.215345 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.214563 10585 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:24.221288 10436 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:24.206125 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 236096us
I20260709 12:10:24.157891 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.155193 10579 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:24.222296 10583 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:24.223145 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.216367 10432 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:24.214828 12171 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:24.227597 12337 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:24.228006 12478 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:24.226796 12171 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:24.230701 12425 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:24.233182 10583 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:24.234851 10579 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:24.231686 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:24.240381 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:24.242928 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.233729 12171 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.243482 12463 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:24.249969 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.233914 12425 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:24.250255 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.247946 12414 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.244019 12413 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.259992 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.260510 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.261524 12464 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:24.263165 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44208us
I20260709 12:10:24.251618 12463 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:24.261996 12387 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.255792 12396 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.279122 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:24.272428 10433 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:24.272346 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:24.280335 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:24.282038 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.280897 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:24.264693 12268 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.286545 12344 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:24.281476 12288 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:24.287900 12415 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.288612 12171 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.284441 12463 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:24.290287 10585 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:10:24.291092 12413 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.302896 12441 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:24.300143 12299 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:24.312222 12441 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:24.312824 12441 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.314846 11212 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:24.310009 12414 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:10:24.294381 12445 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.316636 12414 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:24.317142 12414 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:24.319573 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.320580 10582 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.295351 12344 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:24.288892 12396 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.302521 11216 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:24.328318 11220 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:24.328609 12396 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:24.302908 12487 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:10:24.330667 12487 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:24.330966 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:24.304843 12268 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:24.332456 12268 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:24.332963 12268 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:24.306079 12171 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.331408 12487 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:24.308774 12464 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:24.309684 10581 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:24.338887 12464 raft_consensus.cc:1068] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:10:24.342267 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.301546 12346 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.295814 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:24.349112 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.335345 12268 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:24.352409 12445 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:24.353545 12445 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:24.353972 12445 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:24.354202 10333 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 220) took 15803 ms (client timeout 15000 ms). Trace:
I20260709 12:10:24.318481 12414 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.306200 12415 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.358886 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.361608 10582 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:24.372911 10585 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:24.373287 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.374039 10433 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:24.373952 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:24.374316 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.375154 10585 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:24.356334 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60628.8us
I20260709 12:10:24.375505 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.376247 10553 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:24.376375 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.355624 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:24.376696 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:24.359323 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.377265 10433 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.377877 10578 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:24.358428 10581 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:24.380208 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:24.383405 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.322434 12344 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.387655 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.388902 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.390177 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:24.391855 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.394835 11178 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:24.395336 10433 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:24.391064 12276 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:24.391467 12339 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:24.396456 12276 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.397039 12339 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.354900 12445 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.397871 12339 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.398809 12445 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.403904 12462 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.404665 12171 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:24.405220 12171 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:24.391698 12415 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.397174 12276 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:24.391894 12491 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:24.357157 12346 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:24.411072 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:24.382951 12396 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:24.338065 12487 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.414476 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.413157 10504 raft_consensus.cc:2376] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5. Re-sending same reply.
I20260709 12:10:24.415577 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.385776 12424 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:24.412911 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.381681 12481 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.392863 12439 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.420857 10553 leader_election.cc:400] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:24.422019 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.422322 10507 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:24.428546 10553 leader_election.cc:403] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:24.416930 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.429388 10512 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 1.
I20260709 12:10:24.406317 12171 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.425626 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:24.430420 10504 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.426489 10508 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
W20260709 12:10:24.354802 10333 rpcz_store.cc:269] 0709 12:10:08.551360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.555210 (+  3850us) service_pool.cc:224] Handling call
0709 12:10:24.354108 (+15798898us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:22.647819 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:22.648777 (+   958us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:22.648839 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:10:22.648869 (+    30us) tablet.cc:662] Decoding operations
0709 12:10:22.649428 (+   559us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:22.649521 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:22.649566 (+    45us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:22.649858 (+   292us) tablet.cc:701] Row locks acquired
0709 12:10:22.649881 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:10:22.650120 (+   239us) write_op.cc:270] Start()
0709 12:10:22.650275 (+   155us) write_op.cc:276] Timestamp: P: 1783599022650051 usec, L: 0
0709 12:10:22.650307 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:10:22.651509 (+  1202us) log.cc:844] Serialized 530 byte log entry
0709 12:10:23.437824 (+786315us) op_driver.cc:464] REPLICATION: finished
0709 12:10:23.650918 (+213094us) write_op.cc:301] APPLY: starting
0709 12:10:23.654460 (+  3542us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:23.654657 (+   197us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:23.679060 (+ 24403us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:23.679563 (+   503us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:23.679861 (+   298us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:23.679934 (+    73us) write_op.cc:312] APPLY: finished
0709 12:10:23.680272 (+   338us) log.cc:844] Serialized 23 byte log entry
0709 12:10:23.680504 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:23.680626 (+   122us) write_op.cc:454] Released schema lock
0709 12:10:23.680677 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:23.681151 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:23.681196 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":55323,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":191514,"apply.run_cpu_time_us":2839,"apply.run_wall_time_us":39703,"num_ops":1,"prepare.queue_time_us":465312,"prepare.run_cpu_time_us":33063,"prepare.run_wall_time_us":262105,"raft.queue_time_us":351716,"raft.run_cpu_time_us":16805,"raft.run_wall_time_us":121844,"replication_time_us":775134,"thread_start_us":934948,"threads_started":4,"wal-append.queue_time_us":415167}]]}
I20260709 12:10:24.431210 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.393533 12299 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:24.429741 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:24.455351 11178 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:24.456133 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:24.386125 12440 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:10:24.433599 12346 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.465059 12288 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.426280 12424 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:24.428664 10505 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:24.444020 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:24.473717 10429 raft_consensus.cc:606] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.474213 10429 raft_consensus.cc:614] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:24.447793 12276 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.434166 10433 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:24.453253 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:24.426862 12344 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:24.476831 10431 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:24.477663 12276 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.468567 10504 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:24.469614 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:24.467324 12424 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.479048 12478 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.484944 12478 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.471911 12481 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.487326 12481 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.490356 12425 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.492203 12425 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:24.492620 12276 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:24.493310 12276 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.493906 12276 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.495114 12383 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:24.495839 12383 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.499872 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.500578 10512 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:24.460551 12362 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.502655 12362 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.467965 12493 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:24.506055 12362 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.459565 12396 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.473610 12440 consensus_queue.cc:1243] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:10:24.512033 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.512889 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.514395 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.515326 10585 raft_consensus.cc:2410] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:24.519836 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.520676 10584 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:24.526391 10430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:24.481868 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1527) took 2072 ms. Trace:
I20260709 12:10:24.480789 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:24.553694 10430 raft_consensus.cc:606] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:24.559908 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.560238 10430 raft_consensus.cc:614] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:24.562135 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.563109 10510 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:24.564378 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.565534 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.565696 12415 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:24.566658 12493 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:24.568470 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.570339 10430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:24.490386 12481 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.569268 10506 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
W20260709 12:10:24.547201 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 89738us
I20260709 12:10:24.498890 12383 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:24.551748 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:24.552570 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.524143 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:24.554203 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73508us
I20260709 12:10:24.553630 10587 rpcz_store.cc:276] 0709 12:10:22.409735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.409870 (+   135us) service_pool.cc:224] Handling call
0709 12:10:22.410162 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.750893 (+1340731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.481518 (+730625us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400178754. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:24.481779 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2793251200}
I20260709 12:10:24.527660 12439 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.551123 12319 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:24.568418 12483 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.586s	user 0.006s	sys 0.000s
I20260709 12:10:24.459649 12463 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:24.598829 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.603746 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.604833 10587 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 2: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:24.607082 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.607218 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.613263 10430 raft_consensus.cc:606] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:24.614856 10430 raft_consensus.cc:614] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:24.619933 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36746 (request call id 143) took 1139 ms. Trace:
I20260709 12:10:24.589318 12478 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.624163 10436 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:24.621399 10580 rpcz_store.cc:276] 0709 12:10:23.480508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.534022 (+ 53514us) service_pool.cc:224] Handling call
0709 12:10:24.619651 (+1085629us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b5400336b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:24.619844 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1193346816}
I20260709 12:10:24.609220 12319 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:24.630671 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.598140 12362 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:10:24.616473 12384 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:24.639928 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 159117us
W20260709 12:10:24.640396 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 127585us
I20260709 12:10:24.645756 10430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:24.646600 10430 raft_consensus.cc:606] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:24.648140 10430 raft_consensus.cc:614] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:24.587309 12299 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.652957 12328 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.653718 12328 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:24.654517 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 146) took 1054 ms. Trace:
I20260709 12:10:24.658517 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:24.660188 12463 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:24.661971 12328 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:24.653045 12319 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:24.663795 10430 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:24.664173 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:24.660234 10437 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:10:24.665766 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.654948 10586 rpcz_store.cc:276] 0709 12:10:23.599742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.832107 (+232365us) service_pool.cc:224] Handling call
0709 12:10:23.832588 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:23.833129 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:23.833158 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:23.842586 (+  9428us) log.cc:844] Serialized 26 byte log entry
0709 12:10:23.930547 (+ 87961us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:23.930611 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.943923 (+ 13312us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:24.472681 (+528758us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:24.654135 (+181454us) raft_consensus.cc:1685] finished
0709 12:10:24.654161 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:24.654440 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3083,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":3290,"spinlock_wait_cycles":52613888,"thread_start_us":38239,"threads_started":1,"wal-append.queue_time_us":79503}
I20260709 12:10:24.669090 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:24.613540 12383 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:10:24.612277 12441 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:24.675516 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:24.679509 10586 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
W20260709 12:10:24.684084 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 90854.1us
W20260709 12:10:24.687575 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46504us
I20260709 12:10:24.702512 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:24.702566 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:24.678341 12328 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:24.719194 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 95896.1us
I20260709 12:10:24.714039 12491 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:24.805433 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 84575.3us
I20260709 12:10:24.736080 12493 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:24.838171 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:24.839890 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55759.7us
I20260709 12:10:24.838717 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:24.840698 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:24.842725 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:24.848349 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 112445us
I20260709 12:10:24.850366 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:24.853252 10553 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:24.860911 10507 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:24.863227 12328 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:24.851907 12346 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
W20260709 12:10:24.962977 12503 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:10:24.988153 10510 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:24.968251 12276 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:24.839474 12383 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.969802 10553 leader_election.cc:400] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:24.972013 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:24.868860 12512 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:24.837324 12299 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:24.975597 12441 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:24.975894 12362 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.976281 12514 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:24.976789 12445 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.577s	user 0.015s	sys 0.012s
W20260709 12:10:24.980182 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 137574us
W20260709 12:10:24.999850 12499 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.259s	user 0.003s	sys 0.005s
W20260709 12:10:24.988930 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 429798us
I20260709 12:10:24.992532 10513 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:24.996781 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:24.997926 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 99225.9us
I20260709 12:10:24.998474 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.021759 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:25.002522 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:24.982247 12344 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.022305 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:24.872380 12491 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:25.017031 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 178997us
I20260709 12:10:24.997529 12328 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:25.029090 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:24.981917 12464 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:25.030329 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.999001 12512 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.035778 12512 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:25.020215 12489 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.753s	user 0.000s	sys 0.010s
I20260709 12:10:25.014415 12519 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.037593 12519 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.037957 12519 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.997249 12424 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:25.018191 12501 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.171s	user 0.000s	sys 0.007s
I20260709 12:10:25.039740 12344 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.018378 12362 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.041476 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.008457 12319 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.042340 12319 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.014205 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.043200 10432 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:25.030867 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:25.030079 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 310) took 1183 ms. Trace:
I20260709 12:10:25.030771 12515 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:25.045058 10509 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:25.042779 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:25.075748 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59087.3us
I20260709 12:10:25.053376 10508 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:25.091944 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.033411 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:25.115504 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 82169.5us
I20260709 12:10:25.042735 12319 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:25.099278 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 237708us
I20260709 12:10:25.103204 12344 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.014410 12299 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:25.105280 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76280.9us
I20260709 12:10:25.033699 12213 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.045255 12445 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.047397 12514 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:25.033825 12478 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:25.008885 12383 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:25.111776 12339 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.713s	user 0.020s	sys 0.001s
I20260709 12:10:25.106781 12362 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.076223 12384 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:10:25.110491 12508 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:25.205670 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 113829us
I20260709 12:10:25.235944 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:25.227645 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:25.232858 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114257us
I20260709 12:10:25.120693 12493 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:25.238379 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.249655 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.243248 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:25.255218 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:25.083021 12328 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.216883 12344 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:25.230456 12339 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.259032 12514 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.219749 12462 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:25.274339 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 154872us
I20260709 12:10:25.228734 12424 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:25.228825 12213 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.071758 12464 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.050108 11212 rpcz_store.cc:276] 0709 12:10:23.843375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.843941 (+   566us) service_pool.cc:224] Handling call
0709 12:10:23.884388 (+ 40447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.315371 (+430983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.015772 (+700401us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54007c1fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:25.026968 (+ 11196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989342080}
I20260709 12:10:25.275995 10510 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:25.279516 12425 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.787s	user 0.019s	sys 0.003s
I20260709 12:10:25.298673 12213 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:25.223291 12445 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.256466 12517 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:25.304508 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:25.305251 10583 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:25.295651 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57386.4us
I20260709 12:10:25.285187 12462 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.259085 12506 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.264612 12508 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:25.262015 12493 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:25.239470 12362 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:10:25.308759 10513 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:25.314569 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1577) took 1205 ms. Trace:
I20260709 12:10:25.308940 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.286384 12509 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.315016 10438 rpcz_store.cc:276] 0709 12:10:24.109340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.273269 (+163929us) service_pool.cc:224] Handling call
0709 12:10:24.273649 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.444074 (+170425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.444133 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:24.444182 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:24.444219 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.314299 (+870080us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540033dbd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:25.314340 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.314507 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391408384}
I20260709 12:10:25.301605 12507 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:25.300021 12276 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.313133 12506 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:25.328385 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73288.2us
I20260709 12:10:25.313858 12498 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:25.318224 12509 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.331920 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:25.330132 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 80609.4us
I20260709 12:10:25.315554 12527 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.339424 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:25.341944 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.345855 10583 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:25.334514 12509 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.355123 12507 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:25.356158 12518 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:25.356949 12518 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:25.358290 12518 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:25.359851 12523 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:10:25.365463 12445 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:25.368502 12527 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:25.371520 12508 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.335567 12498 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.406826 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 136) took 1034 ms. Trace:
I20260709 12:10:25.407392 10436 rpcz_store.cc:276] 0709 12:10:24.372190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.372291 (+   101us) service_pool.cc:224] Handling call
0709 12:10:24.372537 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.624738 (+252201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.406472 (+781734us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400436f54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:25.406737 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578180480}
I20260709 12:10:25.408793 12498 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:25.382299 12528 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:10:25.384385 12383 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.386961 12493 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:25.462430 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 118178us
I20260709 12:10:25.388974 12299 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:25.391135 12464 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.400305 12339 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:25.402042 12328 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:25.331874 12506 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:25.410436 12492 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.836s	user 0.003s	sys 0.006s
W20260709 12:10:25.459695 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 150817us
W20260709 12:10:25.467470 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 442039us
W20260709 12:10:25.469225 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 419235us
I20260709 12:10:25.468604 12531 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:25.481057 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.470026 12403 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:10:25.537220 12333 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:25.478542 12519 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.464443 12440 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 12:10:25.534338 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51225us
W20260709 12:10:25.539856 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58949.3us
I20260709 12:10:25.549938 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.550526 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:25.482519 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:25.551563 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 209672us
I20260709 12:10:25.551964 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:25.555711 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84425.5us
I20260709 12:10:25.556490 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:25.537813 12529 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:10:25.539674 12339 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.537793 12333 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:25.557222 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74816.7us
I20260709 12:10:25.478369 12512 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.588819 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.484692 12530 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.490639 12514 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:10:25.549609 12384 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:25.591601 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.590145 12527 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.592631 12413 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.559389 12538 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:25.593348 12508 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:25.592604 12328 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.596088 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:25.616498 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:25.600144 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1555) took 1506 ms. Trace:
I20260709 12:10:25.600164 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.564568 12440 consensus_queue.cc:1243] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:25.547187 12516 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.491129 12540 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 12:10:25.623122 12346 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:25.613171 12384 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:25.623617 12346 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:25.624032 12384 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.624809 12299 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:25.550415 12319 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.595319 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.592880 12464 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:25.618973 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:25.596078 12541 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:10:25.622500 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66126.7us
I20260709 12:10:25.624122 12346 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.620090 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.604689 12507 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:25.612749 12527 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.631497 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.614328 12538 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.618852 12328 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:25.627595 12267 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:25.624051 12519 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:25.618458 12508 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:25.628372 12531 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:25.644008 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:25.628835 12537 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:25.674451 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:25.676539 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:25.635663 12333 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.635810 12540 consensus_queue.cc:1243] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:25.678226 12540 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:25.615025 12413 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.609172 12339 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.637676 12527 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.677465 12213 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:25.683133 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:25.637320 12545 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:25.681457 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:25.645833 12328 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.680554 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:25.639865 12538 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.614010 12530 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:25.685021 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:25.708220 12545 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:25.643327 12445 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:25.683919 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.679152 12276 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.685570 12213 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:25.686658 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:25.691242 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 313) took 1141 ms. Trace:
I20260709 12:10:25.693006 12339 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:25.696426 12535 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:10:25.679955 12540 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:25.699250 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:25.783501 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 112577us
I20260709 12:10:25.784817 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.785722 10437 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:25.785804 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:25.785909 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:25.786686 10436 raft_consensus.cc:2368] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:25.786962 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.787499 12171 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:25.787611 10429 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:25.787940 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:25.688369 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.788096 12171 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:25.788543 10435 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.787611 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.789945 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:25.791076 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.684701 12546 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:25.733173 12553 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:25.618259 10434 rpcz_store.cc:276] 0709 12:10:24.093592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.112725 (+ 19133us) service_pool.cc:224] Handling call
0709 12:10:24.191561 (+ 78836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.253151 (+ 61590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.255843 (+  2692us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:24.268133 (+ 12290us) log.cc:844] Serialized 26 byte log entry
0709 12:10:24.383670 (+115537us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:24.384808 (+  1138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.491931 (+107123us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b5400094a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:24.492016 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:25.409475 (+917459us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:25.599889 (+190414us) raft_consensus.cc:1685] finished
0709 12:10:25.599921 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.600072 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2575,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"spinlock_wait_cycles":71671808,"thread_start_us":41734,"threads_started":1,"wal-append.queue_time_us":191102}
I20260709 12:10:25.797415 12553 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:25.798429 12553 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.799870 12553 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.773077 12507 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.789142 12171 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.733419 12333 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.735011 12504 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:25.738283 12554 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.811242 12504 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:25.811815 12504 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.811451 12554 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:25.731891 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44025.4us
W20260709 12:10:25.736842 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 136750us
I20260709 12:10:25.733276 12299 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:25.746078 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 127189us
I20260709 12:10:25.746953 12487 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.747810 12515 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:10:25.753022 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.753839 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 126) took 1197 ms. Trace:
I20260709 12:10:25.835646 10507 rpcz_store.cc:276] 0709 12:10:24.559326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.559538 (+   212us) service_pool.cc:224] Handling call
0709 12:10:25.753591 (+1194053us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54000daa54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:25.753752 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690233728}
I20260709 12:10:25.755499 12414 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.758232 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:25.759326 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.761442 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.723515 12545 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:25.784845 12337 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:25.700155 12512 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.686748 12387 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:25.789949 10438 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:25.793094 10433 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:25.835393 12515 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:25.677013 12267 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.794342 12213 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.776520 11215 rpcz_store.cc:276] 0709 12:10:24.549653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.551391 (+  1738us) service_pool.cc:224] Handling call
0709 12:10:25.538345 (+986954us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:25.561939 (+ 23594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:25.593132 (+ 31193us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:25.618144 (+ 25012us) log.cc:844] Serialized 14 byte log entry
0709 12:10:25.672170 (+ 54026us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:25.675284 (+  3114us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:25.682004 (+  6720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.686353 (+  4349us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.691151 (+  4798us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9222,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":7354,"raft.queue_time_us":9523,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8630,"spinlock_wait_cycles":3821952,"thread_start_us":16048,"threads_started":1,"wal-append.queue_time_us":54510}
I20260709 12:10:25.783159 12508 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.814792 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.836581 12339 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:10:25.740861 12531 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:25.818575 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.859524 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.821470 10431 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:25.823170 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:25.875250 12319 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:25.852195 12337 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:25.886920 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 63903.3us
I20260709 12:10:25.825817 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.892649 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 142) took 1290 ms. Trace:
I20260709 12:10:25.887418 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.831957 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:25.894801 10431 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
W20260709 12:10:25.895362 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 63493.8us
W20260709 12:10:25.824585 10479 connection.cc:256] RPC call timeout handler was delayed by -1.3678130309999688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2969s for a response
I20260709 12:10:25.723920 12491 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 12:10:25.832916 12445 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.848214 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.848145 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.845872 11218 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:25.854216 12464 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:25.845894 11220 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:25.777200 12276 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.840881 12440 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.838151 12530 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:25.860216 10509 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:25.845656 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:25.869791 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 55147.1us
I20260709 12:10:25.816167 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.872192 12267 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.924605 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:25.925326 11219 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 1.
I20260709 12:10:25.926002 10553 leader_election.cc:400] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:25.926493 10553 leader_election.cc:403] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:10:25.872910 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:25.874905 12509 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.000s
I20260709 12:10:25.889576 11215 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 LEARNER]: Advancing to term 4
I20260709 12:10:25.877207 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:25.887567 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.826535 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:25.895814 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:25.874347 12346 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:25.897557 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:25.894299 10432 rpcz_store.cc:276] 0709 12:10:24.601964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.602164 (+   200us) service_pool.cc:224] Handling call
0709 12:10:24.602601 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:25.043637 (+441036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.892353 (+848716us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400435b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:25.892572 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1012777856}
I20260709 12:10:25.832255 10435 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:25.867658 11216 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:25.904837 12512 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:25.914649 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:25.915048 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:25.919272 12530 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:25.883486 12425 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:25.888995 12384 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.920219 12403 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:25.981506 10583 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:25.920683 12554 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:25.917652 12508 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.890100 12413 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:25.923434 10434 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:25.889483 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:25.887004 12319 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:25.916813 12440 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.928789 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:25.903507 12415 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.820380 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:26.016837 10436 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:25.905892 12495 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.024s	sys 0.032s
I20260709 12:10:26.019600 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:25.992660 11217 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:25.903964 12556 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:25.970883 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 73459.7us
I20260709 12:10:26.028650 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:25.891556 12424 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:25.970757 12537 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:25.974491 11220 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.863744 12506 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:25.888032 12514 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:25.979359 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:26.044299 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65101.3us
I20260709 12:10:25.980624 10587 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:25.913074 12445 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.005017 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:26.006417 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:25.974885 12383 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.008261 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:25.913672 12464 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.013494 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:26.014240 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:26.020345 10580 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:25.969636 10437 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.980775 12563 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:25.983052 12530 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:26.058125 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:26.032033 10431 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:26.019821 12495 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.024s	sys 0.032s
I20260709 12:10:26.009217 12554 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.045151 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:25.970718 12509 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:25.983152 12425 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:26.048254 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 119584us
I20260709 12:10:26.067674 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:26.068549 10507 raft_consensus.cc:2368] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5.
I20260709 12:10:25.903267 12540 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.072225 10580 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:26.053838 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:26.055375 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:26.050104 10586 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:26.074671 10512 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.888175 12531 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.062917 10585 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:25.925026 12538 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.067018 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:26.077149 10513 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:26.078465 10586 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:26.083081 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:26.018862 12319 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.083691 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:26.043339 12514 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.087111 12276 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:26.047765 12552 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:26.047955 12057 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.051689 12516 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:26.103581 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:26.055593 12413 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:26.024371 12415 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.098299 10437 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:26.069324 12333 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.109390 12333 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.112380 12333 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:26.077711 12463 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:26.043733 12537 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.121739 10433 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:26.122171 12537 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.089756 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:26.036698 12556 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.099938 12514 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:26.091836 12538 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:26.134081 12557 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:26.140442 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:26.107621 12413 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.975358 12515 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.141100 12557 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.125506 12463 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.149354 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:26.084904 12554 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.151870 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:26.137738 12550 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:10:26.134774 12516 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:26.155189 10583 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:26.187924 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.047887 12544 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:10:26.149403 12413 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:26.145263 12337 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.220292 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:26.221179 10580 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.221813 10512 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:26.128160 12552 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.230684 10584 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:26.241791 12506 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.141302 12534 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:10:26.142433 12564 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.155422 12463 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:26.165472 12516 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.161499 12537 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:26.166034 12493 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:26.218302 12462 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.277369 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:26.278007 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:26.278662 10506 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:26.267692 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 184142us
I20260709 12:10:26.266395 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:26.217526 12346 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:26.287353 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:26.290975 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:26.290171 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 394089us
I20260709 12:10:26.292829 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:26.297813 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:26.300316 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.147841 12530 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.267704 12516 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:26.265044 12564 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.301046 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:26.287250 12573 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.320096 12573 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.323984 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:26.324663 12573 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:26.264521 12403 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:10:26.284324 12440 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:26.307592 12562 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.154s	user 0.001s	sys 0.001s
I20260709 12:10:26.287671 12384 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:26.288591 12462 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.299549 12517 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:26.298678 12387 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:26.293592 12383 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:10:26.273670 12506 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:10:26.381351 12582 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.316438 12337 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.318955 12564 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.336091 12171 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:26.382714 12507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.027s	sys 0.017s
I20260709 12:10:26.441480 10583 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:26.279891 12493 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.274011 12537 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.367444 12396 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:26.348410 12445 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.358115 12276 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:26.469772 12566 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
I20260709 12:10:26.440461 12333 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.489694 10580 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:26.463244 12582 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:26.464934 12507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.028s	sys 0.018s
I20260709 12:10:26.395087 12425 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:26.504184 10586 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:26.459728 12319 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:26.512579 12579 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.481830 12396 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.470779 12415 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.486008 12509 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:26.485818 12387 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260709 12:10:26.534518 12560 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.354s	user 0.000s	sys 0.002s
I20260709 12:10:26.481709 12424 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.530411 12554 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:26.441118 12462 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.467459 12552 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:26.463685 12171 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.501361 12362 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:10:26.490149 12569 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.541595 12462 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.529492 12556 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:26.542596 12574 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:10:26.466292 12288 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:10:26.510859 12582 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.511389 12333 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.564095 12557 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:26.554646 12328 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.830s	user 0.015s	sys 0.000s
I20260709 12:10:26.514267 12445 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.494489 12276 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.550462 12464 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:26.567514 11220 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.003s
W20260709 12:10:26.572660 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 225505us
I20260709 12:10:26.579808 11220 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:26.544091 12579 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.566677 12507 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:26.617349 12527 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.909s	user 0.009s	sys 0.007s
W20260709 12:10:26.637250 12592 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:10:26.653414 12568 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:26.654186 12568 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:26.623087 12527 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:10:26.890023 10437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.007s	sys 0.009s
I20260709 12:10:26.847791 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:26.843045 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1316) took 1017 ms. Trace:
W20260709 12:10:26.800112 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.909258 10509 rpcz_store.cc:276] 0709 12:10:25.825690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.825838 (+   148us) service_pool.cc:224] Handling call
0709 12:10:25.840191 (+ 14353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:25.913013 (+ 72822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:26.842283 (+929270us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5400228dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:26.842742 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45518080}
I20260709 12:10:26.910239 12333 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:26.802781 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.912613 12396 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:26.913007 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1586) took 1091 ms. Trace:
I20260709 12:10:26.835778 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:10:26.914526 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79520.6us
I20260709 12:10:26.917717 12591 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.618799 12507 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.913630 10436 rpcz_store.cc:276] 0709 12:10:25.821970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.023335 (+201365us) service_pool.cc:224] Handling call
0709 12:10:26.023681 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:26.023937 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:26.023971 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:26.030921 (+  6950us) log.cc:844] Serialized 26 byte log entry
0709 12:10:26.435043 (+404122us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:26.435091 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:26.435182 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:26.798609 (+363427us) raft_consensus.cc:1685] finished
0709 12:10:26.798659 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:26.912712 (+114053us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":365266,"threads_started":1,"wal-append.queue_time_us":435677}
W20260709 12:10:26.907541 10437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.008s	sys 0.009s
W20260709 12:10:26.729734 12553 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.929s	user 0.008s	sys 0.006s
W20260709 12:10:26.678704 12589 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:10:26.622006 12517 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:26.948324 12574 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.328s	user 0.014s	sys 0.030s
W20260709 12:10:26.949115 12545 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.051s	user 0.010s	sys 0.011s
I20260709 12:10:26.950245 12524 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:26.829253 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:26.681759 12567 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:10:26.944124 10479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.006s	sys 0.020s
I20260709 12:10:26.683115 12569 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:26.962390 10479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.007s	sys 0.021s
I20260709 12:10:26.967453 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1589) took 1143 ms. Trace:
I20260709 12:10:26.969398 12579 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:26.974118 12514 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.828s	user 0.014s	sys 0.002s
I20260709 12:10:26.585634 12445 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.673697 12537 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:26.981702 12548 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:26.950182 12545 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:26.951553 12524 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:26.991079 12557 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:26.996650 10584 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:26.999461 12567 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:26.919391 12591 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:26.582834 12328 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:27.017463 11176 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.009s	sys 0.008s
W20260709 12:10:27.024048 10506 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.012s
I20260709 12:10:27.024158 12171 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.024837 10506 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:27.027038 12493 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.028373 10582 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:27.029012 12548 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:27.029496 12504 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.176s	user 0.011s	sys 0.006s
W20260709 12:10:27.030659 11176 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.010s	sys 0.008s
W20260709 12:10:27.025202 11215 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.054s	user 0.010s	sys 0.016s
I20260709 12:10:27.050197 11215 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:27.050930 11215 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 2
I20260709 12:10:27.051353 11215 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 12:10:27.013134 10553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.004s	sys 0.010s
I20260709 12:10:27.051852 11215 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:27.060202 12276 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.061126 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 322) took 1185 ms. Trace:
I20260709 12:10:27.061676 11215 rpcz_store.cc:276] 0709 12:10:25.875846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.883428 (+  7582us) service_pool.cc:224] Handling call
0709 12:10:25.889404 (+  5976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.060578 (+1171174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.061013 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:27.063293 12598 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:27.066215 10553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.005s	sys 0.010s
W20260709 12:10:27.065902 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 237269us
I20260709 12:10:27.064673 10512 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.002878 12445 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:27.018688 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 719545us
W20260709 12:10:27.018560 10432 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.010s	sys 0.025s
W20260709 12:10:27.053704 12564 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.586s	user 0.016s	sys 0.005s
I20260709 12:10:27.080587 12587 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.083642 12563 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.049609 12567 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:27.079741 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.082396 12414 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.236s	user 0.014s	sys 0.005s
I20260709 12:10:27.007395 12514 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.087368 12569 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.087903 12594 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.030159 12504 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:27.095396 10432 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.010s	sys 0.026s
I20260709 12:10:27.086237 11219 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:27.088604 12588 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.086388 12171 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:27.089591 12534 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:10:27.071065 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62463.5us
I20260709 12:10:26.978873 10437 rpcz_store.cc:276] 0709 12:10:25.828648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.112137 (+283489us) service_pool.cc:224] Handling call
0709 12:10:26.112517 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:26.112816 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:26.286341 (+173525us) log.cc:844] Serialized 14 byte log entry
0709 12:10:26.966727 (+680386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:26.966868 (+   141us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:26.966964 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:26.967104 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:26.967371 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10819,"prepare.queue_time_us":414507,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3606,"raft.queue_time_us":12183,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":229,"thread_start_us":820444,"threads_started":2,"wal-append.queue_time_us":677588}
W20260709 12:10:27.071601 10552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.002s	sys 0.039s
I20260709 12:10:27.064224 12598 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:27.095978 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.017s	sys 0.020s
I20260709 12:10:26.679190 12568 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.090909 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:27.099076 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.094560 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:27.106168 10552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.003s	sys 0.040s
W20260709 12:10:27.105867 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.018s	sys 0.020s
I20260709 12:10:27.030606 12548 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.098969 12464 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.082458 12557 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.116298 10434 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:10:27.108268 12534 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:27.119504 10434 pending_rounds.cc:85] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:27.119582 12534 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:27.119956 10434 pending_rounds.cc:107] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:27.120352 10434 raft_consensus.cc:2889] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:27.099426 12564 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.120558 12545 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.126806 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.127518 12568 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.127738 10584 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:27.128065 12591 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:27.123129 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 825310us
I20260709 12:10:27.131835 12527 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:27.137745 11177 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.603s (SENT)
I20260709 12:10:27.138146 12524 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.141244 10512 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:27.149109 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58425.1us
I20260709 12:10:27.153942 12588 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.146273 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.154874 12598 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.154404 10587 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:27.157063 12588 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.158313 10578 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:10:27.159775 12507 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.128551 12587 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.146662 12594 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.174229 12267 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:10:27.178385 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.116619 12517 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.130164 12563 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.183446 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.166639 12268 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.174530 12328 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.196115 12534 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.166666 12598 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:10:27.180222 12587 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.197788 12439 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.214293 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:27.209846 12597 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:10:27.221026 11220 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:10:27.225660 12413 consensus_meta.cc:287] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.965s	user 0.016s	sys 0.006s
I20260709 12:10:27.226262 12413 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.227386 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.228615 10578 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:27.230273 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:27.231127 11220 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.232956 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:27.240206 12587 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.242645 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.234076 12413 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:27.249114 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 64930.8us
I20260709 12:10:27.249492 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:27.219983 12514 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:27.233747 12508 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.223s	user 0.023s	sys 0.003s
I20260709 12:10:27.217994 12439 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.219604 12519 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:27.257620 12519 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.258111 12519 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.267940 12495 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:27.268548 12495 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:27.268944 12495 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.271659 11220 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
W20260709 12:10:27.273288 12463 consensus_meta.cc:287] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.009s	user 0.014s	sys 0.007s
I20260709 12:10:27.295367 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 164) took 1629 ms. Trace:
W20260709 12:10:27.292393 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.276717 12602 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:10:27.300299 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:27.277272 10587 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:27.282562 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.282392 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.280092 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:27.303594 10582 rpcz_store.cc:276] 0709 12:10:25.666427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.666593 (+   166us) service_pool.cc:224] Handling call
0709 12:10:25.681560 (+ 14967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.028919 (+1347359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.295022 (+266103us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54001caa54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:27.295279 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3096008448}
I20260709 12:10:27.288990 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.290104 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.366842 10430 raft_consensus.cc:606] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:27.369874 10430 raft_consensus.cc:614] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:27.284369 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.359262 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:27.283397 12564 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.375917 12569 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.376933 10438 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:27.377550 10429 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:27.377861 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.378912 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:27.380152 12487 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.541s	user 0.013s	sys 0.013s
I20260709 12:10:27.283160 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.293198 12594 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.394677 12609 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.395484 12609 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.395956 12268 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:27.368227 12582 consensus_meta.cc:287] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.794s	user 0.013s	sys 0.012s
I20260709 12:10:27.399551 12582 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.401293 10506 raft_consensus.cc:2368] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:27.383948 12519 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.403378 10505 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
W20260709 12:10:27.385882 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 85862us
I20260709 12:10:27.389642 12362 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:27.390748 10513 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:10:27.394842 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.384393 10429 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:27.405787 10429 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:27.406064 10429 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:27.391806 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1322) took 1115 ms. Trace:
I20260709 12:10:27.277504 12534 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.406836 10512 rpcz_store.cc:276] 0709 12:10:26.276762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.276993 (+   231us) service_pool.cc:224] Handling call
0709 12:10:27.391542 (+1114549us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b540036ef54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:27.391716 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786839808}
I20260709 12:10:27.402664 12582 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.413655 12594 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.413862 12356 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:27.413655 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.416425 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:27.408844 12362 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:27.401322 12515 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.418915 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.357261 12463 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.425872 10508 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
W20260709 12:10:27.428189 12516 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.083s	user 0.011s	sys 0.011s
I20260709 12:10:27.412626 12268 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:27.442041 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.420598 12609 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.453653 12463 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.455152 10513 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:27.455952 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1580) took 1638 ms. Trace:
I20260709 12:10:27.430415 12356 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.456424 10429 rpcz_store.cc:276] 0709 12:10:25.817150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.817868 (+   718us) service_pool.cc:224] Handling call
0709 12:10:25.818232 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.425422 (+1607190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.455631 (+ 30209us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540034a654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:27.455849 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3584212736}
I20260709 12:10:27.459794 12527 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.460166 12516 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.460563 12527 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.463266 12579 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.463490 12555 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:27.459820 12557 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.464720 12557 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:27.464226 12579 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.464468 12555 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:27.465377 12557 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.444094 12561 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.467413 12561 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:27.444676 12506 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.445855 10510 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:10:27.469273 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:27.446259 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 195096us
I20260709 12:10:27.469305 12506 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.446391 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:27.471757 10509 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:27.472018 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:27.474349 12506 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.480127 12557 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.480382 12487 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:27.446903 12588 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.459240 12396 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.442727 10507 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.470656 12299 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:27.429811 12587 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.463153 12564 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.486829 12564 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.487131 12396 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.489146 12564 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.489495 12396 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.489998 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.482439 12588 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.490659 10582 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:27.434247 12328 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.475201 12527 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.462735 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:27.481451 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 98989.1us
I20260709 12:10:27.485086 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.485306 12579 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.485847 12587 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.493674 12569 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.493896 12588 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.493580 12328 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.482479 12463 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.462939 12516 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.500228 12527 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:27.496908 12413 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.429025 12495 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.482784 12519 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.503430 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.508172 10506 raft_consensus.cc:2393] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:27.505000 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.500584 12463 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.513007 12538 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.513731 12538 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:27.511271 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1587) took 1688 ms. Trace:
I20260709 12:10:27.515755 10434 rpcz_store.cc:276] 0709 12:10:25.822383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.044811 (+222428us) service_pool.cc:224] Handling call
0709 12:10:26.088448 (+ 43637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.175540 (+1087092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.508124 (+332584us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400087ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:27.511178 (+  3054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2325071744}
I20260709 12:10:27.512708 12288 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:10:27.506599 12563 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.520222 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1584) took 1700 ms. Trace:
I20260709 12:10:27.513919 10336 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:27.509248 12527 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:27.506574 12413 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.536978 12504 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.506063 12555 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.518887 12288 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:27.503244 12524 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.519236 12268 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:27.579497 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46255.4us
I20260709 12:10:27.594640 10507 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:27.597337 12356 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.600666 12524 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.604797 12504 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.605197 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:27.605703 12504 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.588734 12276 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.607283 12520 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.607821 12276 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.608152 12520 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.610493 12276 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.617766 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:27.610739 12520 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.636216 12463 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.792210 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.683327 12563 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
W20260709 12:10:27.687932 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.793411 10587 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:27.634600 11212 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:27.580228 10432 rpcz_store.cc:276] 0709 12:10:25.820287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.968039 (+147752us) service_pool.cc:224] Handling call
0709 12:10:25.978957 (+ 10918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:25.979380 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:27.111076 (+1131696us) log.cc:844] Serialized 14 byte log entry
0709 12:10:27.373696 (+262620us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.495110 (+121414us) log.cc:844] Serialized 26 byte log entry
0709 12:10:27.498079 (+  2969us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:27.498130 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.500041 (+  1911us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:27.519991 (+ 19950us) raft_consensus.cc:1685] finished
0709 12:10:27.520011 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:27.520148 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":487203,"prepare.run_cpu_time_us":1730,"prepare.run_wall_time_us":12457,"raft.queue_time_us":72566,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":24183,"spinlock_wait_cycles":271616,"thread_start_us":1193192,"threads_started":2,"wal-append.queue_time_us":301180}
I20260709 12:10:27.687723 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.719493 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:27.629731 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.634286 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50250 (request call id 20) took 2970 ms. Trace:
I20260709 12:10:27.628700 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.623085 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.637873 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.797634 10582 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
W20260709 12:10:27.710525 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 203332us
I20260709 12:10:27.630344 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.628232 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.788534 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:27.623529 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.809607 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:27.810529 10438 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.811321 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.812322 10505 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:27.720276 12519 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:27.813980 11219 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:27.815352 10504 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:27.817723 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.818517 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.820024 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.797942 10430 rpcz_store.cc:276] 0709 12:10:24.663454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.663643 (+   189us) service_pool.cc:224] Handling call
0709 12:10:27.633955 (+2970312us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b540034a654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ad9853 00007f2c97aa760a 00007f2c9102d7d4 00007f2c9102d762 00007f2c9102d691 00007f2c9102d60c 00007f2c9102c8c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:27.634148 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6214546176}
W20260709 12:10:27.790911 12617 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.290s	user 0.002s	sys 0.000s
I20260709 12:10:27.624945 12495 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.598961 12538 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.610795 12413 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.623937 12524 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.637290 12328 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.794761 12582 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.846490 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.622857 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:27.720062 10336 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:27.852219 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.858505 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.737939 12548 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:27.854686 10513 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:27.855160 10508 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
W20260709 12:10:27.877439 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.619215 12276 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.622521 12579 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:27.857445 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:27.859822 11220 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.860376 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.860491 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.861057 10587 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:27.861629 10434 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.862200 10436 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:27.862716 10430 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:27.864073 10580 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:27.864542 10336 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:27.867242 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.814613 12519 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.868016 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.868146 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:27.892501 10509 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:27.868793 12491 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:27.878394 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.879073 10506 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
W20260709 12:10:27.882067 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 397939us
I20260709 12:10:27.889614 10432 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:27.891042 10430 pending_rounds.cc:85] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:27.898890 10430 pending_rounds.cc:107] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:27.873260 12548 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:27.885454 12579 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.882247 12561 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.899289 10430 raft_consensus.cc:2889] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:27.886849 12587 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.910148 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:27.907292 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:27.720794 12362 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.910012 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.914348 10586 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:27.916204 11177 leader_election.cc:400] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5.
I20260709 12:10:27.916558 11177 leader_election.cc:403] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:27.893579 12491 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:27.900648 12508 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:27.917811 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:27.918150 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:27.890946 12527 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.907703 12514 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:27.882436 12582 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.919373 12563 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:27.908787 12579 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:27.929121 10581 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:27.930298 10504 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:27.929543 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.941669 10582 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:27.920655 10580 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:27.946370 10578 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:27.920738 10435 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:27.929940 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.934834 11219 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:27.947190 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:27.934773 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 86346.1us
I20260709 12:10:27.948028 10432 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:27.958163 12548 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.959045 12563 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.959676 12548 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.961761 12563 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.938699 12595 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:27.963351 12567 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.963634 12595 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:27.966250 12595 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.938380 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:27.976190 10438 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:27.946856 12530 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.016s	sys 0.023s
I20260709 12:10:27.939105 12491 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:27.931450 12362 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.982754 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:27.983105 12346 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.983573 10580 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:27.984077 12491 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.944994 12504 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:27.943756 12515 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:27.940388 12519 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.934477 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:27.937465 12582 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:27.989600 12328 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:27.936993 12439 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.938318 12506 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.994140 10583 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:27.981981 12530 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.016s	sys 0.024s
I20260709 12:10:27.935360 12495 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5.
I20260709 12:10:27.934160 12514 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.003585 10336 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:28.004557 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:27.997865 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.920511 12276 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:27.995463 12618 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.383s	user 0.000s	sys 0.002s
I20260709 12:10:28.009554 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 4, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:27.999571 12506 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.038235 12506 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.039700 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.043264 10336 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 4, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:28.045284 10336 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 4, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:28.046873 10336 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:10:28.004606 12595 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.052109 10583 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:28.053242 10336 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:28.054473 12276 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.054517 11220 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:28.054827 10336 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 4, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:28.055951 10438 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.056330 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.057374 10579 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:28.058269 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.059401 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.060039 10579 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:28.061509  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:27.994890 12515 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.032610 10436 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
W20260709 12:10:28.077075 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 77030.9us
I20260709 12:10:28.082166 10438 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:28.088032 11212 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 284) took 1736 ms (client timeout 1916 ms). Trace:
I20260709 12:10:28.090688 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.093052 12595 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.096208 12514 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:27.994571 12508 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:28.098426 12515 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:28.120076 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:28.099140 12623 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:10:28.104766 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:28.109109 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 170927us
I20260709 12:10:28.112145 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:28.105017 12534 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.000s
I20260709 12:10:28.120012 10512 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:28.126710 12508 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.120852 10585 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:28.123699 10579 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:28.106873 10583 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:28.136731 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.147280 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:28.136526 12487 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:28.139106 12587 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.136750 12356 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.162726 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:28.144382 12439 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:28.112308 11212 rpcz_store.cc:269] 0709 12:10:26.351561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.354707 (+  3146us) service_pool.cc:224] Handling call
0709 12:10:26.847286 (+492579us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:28.087745 (+1240459us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b5400591854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:28.087901 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379938432}
I20260709 12:10:28.152136 12561 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:28.159051 10505 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:10:28.169659 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 214761us
I20260709 12:10:28.179875 12534 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.180094 12356 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.179481  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:28.178809 12561 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.171478 12487 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:28.205262 10585 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:28.197517 11217 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:10:28.182906 12356 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.215176 10583 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:28.204144 12268 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:28.236618 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.212744 12524 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:28.238924 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 102499us
I20260709 12:10:28.243752 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.235481 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:28.189824 12587 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.252454 10581 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:28.208828 12439 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.256438 11221 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
W20260709 12:10:28.254891 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:28.244861 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.260072 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:28.261709 10584 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.263121 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:28.263767 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.264879 10434 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:28.265239 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.265975 10507 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:28.266423 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:28.267352 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44772.9us
I20260709 12:10:28.267988 10509 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:28.268261 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.269619 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.271292 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:28.270569 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.273141 12463 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.273590 12463 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.273958 12463 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.262598 12524 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.277037 12524 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.277645 12524 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:28.278030 12524 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:28.278430 12524 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:28.281023 12439 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:28.282140 10584 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:28.282913 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.261539 12515 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:10:28.283877 12439 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.284317 12439 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.284358 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:28.284821 12439 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:28.285156 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.285827 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.286319 10583 raft_consensus.cc:2368] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:28.270529 12413 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.294821 12413 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:28.295372 12413 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:28.295475 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.297313 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:28.297890 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.299139 12595 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:28.289598 12356 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:28.300432 12595 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.301129 12595 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:28.301638 12595 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:28.194876 12487 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.287364 12268 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.310290 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:28.312014 12534 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:28.315236 12520 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:28.316362 12520 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:28.316838 12520 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:28.317745 12520 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:28.319869 12625 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:10:28.321821 12588 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.322297 12588 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.322623 12588 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.308184 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.325646 10509 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:28.306013 12276 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.337404 10580 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:10:28.342464 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.307163 12561 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.292332 12463 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.358438 10504 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:28.356531 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.364650 10509 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:28.361389 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:28.373628 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:28.378572 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.347414 12520 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.388533 10507 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:28.344892 12527 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:28.395509 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.396448 10509 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:28.396230 12587 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.410022 10509 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:28.386864 12362 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.400588 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:28.415372 10581 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:28.416085 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 130326us
I20260709 12:10:28.418452 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:28.360154 12534 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 12:10:28.407904 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.399107 11215 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:28.384560 12495 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:10:28.398442 12516 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:28.428227 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.428969 10584 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:28.417012 12527 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.423451 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:28.430084 10578 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.435714 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:28.436404 10584 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:10:28.436302 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 100319us
W20260709 12:10:28.437727 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98018.4us
I20260709 12:10:28.440217 12517 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:28.442111 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:28.442721 11212 raft_consensus.cc:2376] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5. Re-sending same reply.
I20260709 12:10:28.445559 10581 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:28.437804 12498 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:28.448717 10581 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:28.454072 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:28.460021 12634 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:10:28.461184  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:28.469482 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:28.470446 11214 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:28.432111 12516 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:28.425321 12356 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:28.476501 12636 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:28.477017 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.487286 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.487807 10508 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:10:28.487996 10587 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:28.488310 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.489411 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:28.490121 10513 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:28.487286 12636 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.490821 12636 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.491225 12516 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:28.493073 10507 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: [3.5-4.6]
I20260709 12:10:28.493563 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.433442 12587 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:10:28.431172 12362 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.479852 10504 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:28.495376 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.482838 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.496348 10510 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:28.496634 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.497134 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.402601 12595 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.498353 12595 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.498579 12515 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:28.499109 12515 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:28.500136 12515 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.501624 12515 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:28.503146 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.503371 12299 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:28.481662 10511 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:28.488209 10508 pending_rounds.cc:85] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 4
I20260709 12:10:28.505097 10508 pending_rounds.cc:107] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:10:28.494840 12276 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:28.471606 12463 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.486219 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.503750 12299 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:28.505443 10508 raft_consensus.cc:2889] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:28.478091 12548 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.002s
I20260709 12:10:28.483424 12579 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.486433 12527 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:28.472303 12498 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:28.519675 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.520592 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 329) took 1341 ms. Trace:
I20260709 12:10:28.522250 10506 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:28.522132 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.551106 10437 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.555379 10510 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:28.519793 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.525038 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.556753 10582 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:28.557092 10504 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:28.531176 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:28.536770 12362 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.549635 11221 rpcz_store.cc:276] 0709 12:10:27.178819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.183102 (+  4283us) service_pool.cc:224] Handling call
0709 12:10:28.520354 (+1337252us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004bee54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:28.520516 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2461917952}
I20260709 12:10:28.540067 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:28.543125 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 109298us
I20260709 12:10:28.559294 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.511281 12276 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:28.574909 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:28.575527 10579 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:28.609567 10584 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:28.609756 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.611246 12587 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:28.611693 12587 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.612053 12587 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.613174 12579 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:28.555565 12268 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.615168 12635 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:28.615377 12268 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.615837 12268 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.615984 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:28.617272 12561 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:28.617660 12561 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.617997 12561 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.619617 12487 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:28.624313 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:28.626106 10437 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:28.616214 12268 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.618351 12413 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.704648 12498 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.707096 12635 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.714535 10435 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:28.715804 12561 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.717808 11221 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:28.722500 10436 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 12:10:28.723431 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 99283.4us
W20260709 12:10:28.689559 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.723173 10436 pending_rounds.cc:85] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 1
I20260709 12:10:28.731266 10436 pending_rounds.cc:107] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:10:28.732203 10436 raft_consensus.cc:2889] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:28.728302 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 200390us
I20260709 12:10:28.521894 12299 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.712105 12439 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:28.730940 11178 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.151s (SENT) [suppressed 18 similar messages]
W20260709 12:10:28.716972 12491 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.002s
W20260709 12:10:28.752080 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.753510 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.753737 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:28.754462 11221 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
I20260709 12:10:28.746781 12561 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:28.716754 12524 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.762389 12527 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.763577 12527 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.757622 12595 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.766098 12595 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.767202 12514 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:28.767583 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.768568 10429 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:28.769060 12595 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:28.769558 12299 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.741391 12356 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.766769 12527 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:28.771544 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.772904 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.773928 12413 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.774034 10429 raft_consensus.cc:2368] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5.
I20260709 12:10:28.775256 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:28.775544 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.776099 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.777593 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.771260 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.777522 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:28.781783 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.778429 10436 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:28.783591 10438 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.780982 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.789569 11219 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:28.790418 10436 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 4. Ops: []
I20260709 12:10:28.791393 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:28.792074 11214 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.795801 11215 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:10:28.796808 12439 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.805913 12276 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.778153 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.808591 10430 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:28.808858 10433 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:28.817670 10504 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:28.822105 12588 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.823709 10434 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.824021 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.826581 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.818760 12587 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.838292 12587 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.828284 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.840543 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.841251 10430 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:28.850344 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.856765 10432 raft_consensus.cc:2368] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:28.857988 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.863543 12516 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:28.858135 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.870106 12516 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:28.851436 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.871555 10434 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:28.872272 12588 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.868049 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:28.875514 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.875802 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.876539 10435 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:28.878098 10435 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:28.878525 10511 raft_consensus.cc:2368] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:28.880255 12524 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.883646 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.884500 10438 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:28.885462 12276 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.885599 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.881353 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.865079 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.887933 10430 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:28.888250 10438 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.889024 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.889243 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.890049 10430 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:28.890959 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.895224 10578 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:28.895838 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.867308 12636 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.892895 10432 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:28.853783 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.895160 10438 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.898799 10509 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:28.895850 10435 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:28.899667 10433 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.900589 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.898896 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.909060 10435 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:10:28.915213 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 153614us
I20260709 12:10:28.924417 10435 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:28.925096 12636 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:28.926046 10431 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:28.927456 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.928284 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.928967 10432 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
W20260709 12:10:28.929718 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.940582 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.942227 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.949610 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:28.951174 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.953200 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.960559 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.961771 10437 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.968740 10435 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:28.970539 10432 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.971691 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.972324 10432 raft_consensus.cc:2376] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4. Re-sending same reply.
I20260709 12:10:28.973376 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:28.954201 10431 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:28.980309 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:28.983567 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:28.984185 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:28.896250 12268 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:28.918810 12587 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:10:28.935163 12527 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:10:28.984652 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:29.063303 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52037.7us
I20260709 12:10:29.024832 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:29.037720 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.085896 10434 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:29.020480 10435 raft_consensus.cc:2368] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
W20260709 12:10:29.020282 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 258615us
W20260709 12:10:29.060869 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.168476 10552 leader_election.cc:400] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:29.168859 10552 leader_election.cc:403] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:10:28.937840 12508 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:29.175272 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 204850us
I20260709 12:10:29.178013 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.181704 10432 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:29.192458 10552 leader_election.cc:400] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:29.192961 10552 leader_election.cc:403] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:29.024750 12515 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:29.203670 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.217999 10438 raft_consensus.cc:2410] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:28.941645 12534 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.243s
I20260709 12:10:29.223608 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.227946 10438 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:29.236091 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 60801.6us
I20260709 12:10:29.183390 12514 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:29.237066 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:29.241210 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.199105 11217 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:29.135728 12506 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:10:29.183992 12579 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:29.250557 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:29.229728 10580 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.008s	sys 0.026s
I20260709 12:10:29.259804 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.263321 12516 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:29.263536 12346 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:29.264288 10431 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:29.264459 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.272835 11214 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.276597 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.284643 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.289115 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.289326 10429 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:29.290112 10434 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:28.991595 12595 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.258s
I20260709 12:10:29.191294 12508 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:29.295224 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.299528 10436 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:29.303514 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.304500 10437 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:29.247939 12362 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:10:29.285933 12514 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:29.293383 12598 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.352768 10435 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:29.340247 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:29.343269 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 148060us
I20260709 12:10:29.353615 10434 raft_consensus.cc:2376] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3. Re-sending same reply.
W20260709 12:10:29.358465 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 121497us
I20260709 12:10:29.362366 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.366060 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.367295 10431 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:29.366904 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:29.368444 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:29.362850 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 180875us
I20260709 12:10:29.369325 10432 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:29.370957 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.371879 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:29.375173 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.376027 10433 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:29.348080 12514 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:29.383270 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.393586 10437 raft_consensus.cc:2368] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:29.350061 12524 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.402275 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:29.403224 10433 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:29.405112 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:29.405129 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 161281us
I20260709 12:10:29.373013 12595 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:10:29.414170 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.418962 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.417583 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:29.422359 10404 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:29.373988 12598 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.427656 10478 leader_election.cc:400] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:29.431097 10478 leader_election.cc:403] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:29.397780 12534 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:10:29.432765 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.436257 10585 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:29.440719 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:29.427990 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:29.432642 12299 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.663s	user 0.015s	sys 0.000s
W20260709 12:10:29.487023 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.426888 12636 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:29.512899 12595 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3.
I20260709 12:10:29.514426 12595 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.410755 12524 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:29.527307 10580 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:29.430387 12661 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:29.517030 12649 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:10:29.439276 12517 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 12:10:29.503436 12491 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:29.563405 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 192563us
W20260709 12:10:29.567322 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.567873 11219 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:10:29.568327 11219 pending_rounds.cc:85] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 5
I20260709 12:10:29.568579 11219 pending_rounds.cc:107] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:10:29.568836 11219 raft_consensus.cc:2889] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:29.436188 12657 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
W20260709 12:10:29.549310 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.525488 12636 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:29.573992 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:29.542219 12587 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:29.550079 12650 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:10:29.550096 12653 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:10:29.555289 12515 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:29.580619 12659 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:29.585709 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.586225 12636 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.573060 12491 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.588517 12508 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:29.599745 12508 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:29.600924 12508 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:29.601116 12491 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:29.601438 12508 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:29.601923 12508 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:29.603070 10553 leader_election.cc:336] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.501s (SENT)
I20260709 12:10:29.540233 12595 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.580967 12356 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:10:29.606225 12595 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:29.608430 12598 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:29.606909 12595 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:29.610219 12595 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:29.611461 12595 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.613358 12595 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.615861 12515 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:29.617802 12524 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:29.633168 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59241.8us
I20260709 12:10:29.634256 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:29.622666 12276 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.000s
I20260709 12:10:29.630378 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:29.579161 12661 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:29.638742 10553 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.531s (SENT)
I20260709 12:10:29.640487 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:29.584756 12587 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:29.649598 10553 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.653s (SENT)
I20260709 12:10:29.590119 12659 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:29.659570 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:29.649614 12664 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:29.664223 12587 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:29.664947 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:29.636173 12548 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:10:29.658191 12520 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.658438 12659 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:29.656675 12598 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:29.670925 10430 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:29.675587 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.664942 12534 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:29.679561 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.665599 12661 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.691345 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:29.693719 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 106732us
I20260709 12:10:29.726662 10578 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
W20260709 12:10:29.684849 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 182964us
W20260709 12:10:29.704411 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:29.709849 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 292344us
I20260709 12:10:29.705214 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:29.717384 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.731948 10584 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:29.717424 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.733212 10581 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.718194 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.733526 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.714965 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.734840 10579 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.689157 12664 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:29.737473 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.713258 12657 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:29.739910 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:29.742679 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.741645 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:29.746214 12299 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:29.746381 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:29.746762 12299 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:29.747329 12299 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.719370 12668 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.749296 10579 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:29.743894 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.750119 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:29.753259 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.727586 12491 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.759153 10587 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:29.755395 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.759886 10578 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.754824 10585 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.757230 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.758345 10584 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:10:29.761828 10584 pending_rounds.cc:85] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:10:29.762197 10584 pending_rounds.cc:107] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:10:29.762519 10584 raft_consensus.cc:2889] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:29.758221 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 93359.4us
I20260709 12:10:29.704227 12663 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:10:29.766258 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.772867 10581 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.781997 12636 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:29.790262 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.748490 12498 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:29.720367 12520 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.794463 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.794657 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.794657 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.748610 12671 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:29.807391 10578 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:29.804466 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.805608 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:29.788640 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:29.802912 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.796595 10581 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:29.799371 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.804816 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:29.812127 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91321.2us
I20260709 12:10:29.805146 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.814836 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:29.825203 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78933us
W20260709 12:10:29.831336 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48715.7us
I20260709 12:10:29.878432 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:29.789466 12660 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:29.852926 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 116368us
I20260709 12:10:29.854405 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.854712 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:29.792512 12506 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:29.859658 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:29.860069 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.859723 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:29.888136 10509 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:29.888520 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:29.863478 10507 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:29.864058 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:29.867195 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.806522 12671 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:29.767452 12491 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.759394 12668 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.811496 10505 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:29.880944 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:29.892495 10581 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:29.873562 12657 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:29.885226 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:29.750033 12664 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.871703 12268 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:29.884493 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:29.889166 10504 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.890380 10513 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:29.890547 10584 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:29.884289 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:29.893899 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:29.895450 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:29.887573 12506 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:29.930878 10508 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:29.900910 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:29.905690 10511 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:29.913740 10581 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:29.915658 12682 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:29.917457 12491 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:29.918210 11215 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:29.918536 12668 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:29.919351 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.949322 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:29.880985 12534 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:29.939013 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 153775us
I20260709 12:10:29.891579 12636 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:29.920933 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.894601 12660 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:29.944144 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 59753.6us
I20260709 12:10:29.894857 12679 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.989987 12346 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:29.990370 12508 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.991102 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:29.946339 12669 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:29.992000 12669 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:29.992424 12669 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:29.992946 12508 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:29.994431 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.893430 12671 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.003927 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.004786 10583 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.005831 12534 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:10:29.962396 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67009.8us
I20260709 12:10:29.968220 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:29.975575 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.008050 10581 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.988480 12660 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:29.941030 12681 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:29.980541 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:29.989857 12679 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:29.995873 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.015626 10578 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:29.935997 12682 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:29.997337 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.017316 10582 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:29.942116 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:29.939050 12680 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:10:29.944386 12587 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:30.007627 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.010336 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.009049 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:30.021903 10584 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:29.977180 12579 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:30.020615 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 78647.3us
I20260709 12:10:30.024134 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:30.024935 10580 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:29.961218 12667 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:10:30.035725 10504 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
W20260709 12:10:30.043604 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66353.9us
I20260709 12:10:30.045552 10552 leader_election.cc:400] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:30.046126 10552 leader_election.cc:403] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:30.048990 12657 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.050681 12664 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:30.051225 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:30.053817 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 85727.9us
W20260709 12:10:30.058223 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:30.059914 11177 leader_election.cc:336] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.888s (SENT)
I20260709 12:10:30.066265 10581 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:30.019243 12682 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.026201 12684 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:30.028723 12681 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:30.070080 12681 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.030331 12517 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.074482 12517 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.061404 12636 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:30.027494 12508 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:10:30.076574 12362 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.076728 12508 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:30.077103 12362 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:30.077450 12508 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.023614 12579 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.079242 12579 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:30.028528 12598 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:30.081414 12598 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:30.081858 12598 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.049201 12506 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.075399 12636 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:30.083232 12636 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:30.083441 12506 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.074913 12517 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.087067 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:30.088954 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56259.9us
I20260709 12:10:30.093446 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.094135 10430 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.097720 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.098404 10430 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:30.103550 10584 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
W20260709 12:10:30.104202 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46959.9us
I20260709 12:10:30.105448 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.106778 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.109941 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.114146 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:30.116628 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:30.117203 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.121402 10583 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:30.114892 10505 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:30.077601 12362 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.127180 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:30.133483 10586 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.127735 12655 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:30.130818 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:30.141049 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:30.131744 10507 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:30.141333 10584 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:30.137512 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78950.9us
I20260709 12:10:30.137908 10432 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:30.131141 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:30.145174 10504 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.146055 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.140489 12655 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.113057 12514 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:10:30.148077 12655 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.148645 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:30.148800 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.129148 12682 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:30.151310 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.070098 12679 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:30.111002 12664 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:30.111466 12670 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:30.156095 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 96261us
I20260709 12:10:30.140828 10510 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:30.159777 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:30.162130 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.163270 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:30.157517 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:30.119189 12680 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:30.161134 11177 leader_election.cc:336] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.563s (SENT)
I20260709 12:10:30.163715 10433 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:30.166014 10583 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.162413 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.165861 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.166378 11221 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.128710 12299 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.172803 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:30.135094 12688 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:30.174862 10584 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:30.167146 10435 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.182833 10585 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:10:30.112823 12516 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:10:30.182710 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:30.125442 12667 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:10:30.159516 12689 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:30.194752 10504 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:30.195043 10579 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:30.200793 10435 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:30.202411 10553 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:30.202762 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.210777 10553 leader_election.cc:403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:30.211066 10438 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:30.211974 12680 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:30.193990 10433 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:30.212278 12664 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:30.202471 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:30.212493 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.213984 12664 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:30.214228 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:30.214491 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.215499 12664 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.217058 10587 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:30.254232 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.256170 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:30.252100 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 90994.7us
I20260709 12:10:30.256202 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:30.155869 12587 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:10:30.264185 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.290740 10582 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:30.191665 12663 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:30.202075 12693 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:30.207999 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:30.216226 12684 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:30.324506 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.252941 12516 consensus_queue.cc:1243] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:10:30.326661 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.274317 12680 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:30.292433 12689 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:30.302423 12670 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:30.301223 12661 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:30.343626 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51920.9us
I20260709 12:10:30.322506 12515 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.319263 12508 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:30.367225 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.353456 12515 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.335882 12680 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:30.377339 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 231328us
W20260709 12:10:30.367168 12659 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.627s	user 0.006s	sys 0.008s
I20260709 12:10:30.339031 11212 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:30.325757 12491 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:10:30.381016 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:30.392198 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.368072 12697 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:30.376679 12692 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:30.401430 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 335) took 1989 ms. Trace:
I20260709 12:10:30.468756 10587 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:10:30.406311 12691 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260709 12:10:30.406690 12694 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:10:30.390352 12268 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:30.407692 12299 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:30.408562 12527 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.417050 12684 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:30.420979 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.382038 12508 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.399497 12587 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:10:30.427042 12700 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:30.375314 12356 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:30.380966 12515 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.385730 12680 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:30.447475 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96326.4us
I20260709 12:10:30.473518 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:30.496551 11220 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:30.429826 12697 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:30.464583 12692 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:30.515187 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:30.566150 10507 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:30.516628 10504 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:30.519243 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:30.569207 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:30.569981 10511 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:30.522516 12656 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:30.497501 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 237569us
I20260709 12:10:30.389694 12659 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.494038 12524 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:30.547042 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:30.548211 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 292013us
I20260709 12:10:30.511083 12527 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.612073 12706 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:30.503228 12514 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.495822 12670 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:10:30.557837 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:30.706354 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:30.572798 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 99442.5us
I20260709 12:10:30.572467 12498 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.574347 12692 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.708279 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:30.630821 12652 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:10:30.534631 12579 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.540377 12508 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.710357 12514 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:30.723410 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:30.724215 11218 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:30.724340 12579 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:30.724407 12508 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:30.724781 12514 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:30.542706 12682 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:30.667985 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 120698us
I20260709 12:10:30.725203 12663 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:30.725718 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.473008 11217 rpcz_store.cc:276] 0709 12:10:28.409833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.420831 (+ 10998us) service_pool.cc:224] Handling call
0709 12:10:28.445307 (+ 24476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:29.283048 (+837741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:29.675949 (+392901us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b540047aca0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae2a02 00007f2c92adedd6 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:10:30.375961 (+700012us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b540047a9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:30.399017 (+ 23056us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1667806208}
I20260709 12:10:30.726716 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:30.727339 10504 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.655342 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 340) took 1152 ms. Trace:
I20260709 12:10:30.622816 12524 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:30.705689 10513 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:30.711362 10511 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:30.545503 12700 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:30.647874 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.672935 12527 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.550482 12356 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:30.733402 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:30.733552 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:30.761157 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:30.762604 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:30.742339 12655 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.002s
I20260709 12:10:30.716156 12697 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:30.746037 12636 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.001s
I20260709 12:10:30.745941 12514 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:30.747124 12696 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:10:30.747865 12595 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:30.672935 12706 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:30.749954 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 241282us
W20260709 12:10:30.742190 12671 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.738s	user 0.005s	sys 0.005s
I20260709 12:10:30.737509 12670 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.762848 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.775236 10432 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.634371 12656 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:30.772895 10478 leader_election.cc:336] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.558s (SENT)
W20260709 12:10:30.791015 11178 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.515s (SENT)
W20260709 12:10:30.823266 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56829.1us
I20260709 12:10:30.779049 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:30.824218 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.824810 10433 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:30.827082 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:30.774700 10435 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:10:30.829885 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57039.5us
I20260709 12:10:30.831476 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.832175 10504 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:30.721308 12498 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:30.771272 12268 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:30.833825 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:30.838171 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:30.769807 12711 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.777724 12670 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:30.772485 12362 consensus_meta.cc:287] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.004s
W20260709 12:10:30.863394 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.769968 12655 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.772042 12701 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:30.882647 12701 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.882539 12508 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:30.882771 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.883175 12701 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:30.880841 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 49438.2us
I20260709 12:10:30.881433 12719 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:30.882076 10511 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:30.885356 12698 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:30.776786 12706 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:30.907779 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:30.752526 12667 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:10:30.772359 12595 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:30.878600 12697 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.773517 12587 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.146s
I20260709 12:10:30.947199 12508 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:30.843339 12711 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:30.819121 12671 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.879418 12670 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.767380 12636 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:30.886113 12679 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:30.907989 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:30.885460 12498 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:30.961621 10585 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:30.885919 12362 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:30.963099 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:30.939357 12698 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.969209 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 344) took 1236 ms. Trace:
I20260709 12:10:30.971773 12671 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:30.973162 11178 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.151s (SENT)
I20260709 12:10:30.731021 11214 rpcz_store.cc:276] 0709 12:10:29.502709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.504966 (+  2257us) service_pool.cc:224] Handling call
0709 12:10:29.505517 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:29.506093 (+   576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:29.506129 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:29.946770 (+440641us) spinlock_profiling.cc:243] Waited 9.76 ms on lock 0x7b2c0001cb68. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:10:29.952024 (+  5254us) log.cc:844] Serialized 26 byte log entry
0709 12:10:30.064799 (+112775us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:30.064872 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.064994 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.405533 (+340539us) spinlock_profiling.cc:243] Waited 43.3 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:30.619737 (+214204us) raft_consensus.cc:1685] finished
0709 12:10:30.625529 (+  5792us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.646463 (+ 20934us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5243,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":194,"spinlock_wait_cycles":122138368,"thread_start_us":106205,"threads_started":1,"wal-append.queue_time_us":158661}
I20260709 12:10:30.977444 10584 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:30.910113 12719 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:30.990293 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 107588us
I20260709 12:10:30.989195 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:30.993690 10578 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:30.962998 12711 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.013579 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:31.018003 10505 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:30.959941 12508 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:30.976686 12636 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:31.025597 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 117935us
W20260709 12:10:31.033740 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 209637us
I20260709 12:10:30.995780 12516 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.037310 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:31.039515 10511 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.039454 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:30.998927 12698 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.006215 12684 raft_consensus.cc:2764] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:10:30.995802 12733 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.011775 12719 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:31.060278 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:31.035121 12669 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.042s	user 0.015s	sys 0.000s
I20260709 12:10:31.092096 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.044755 12657 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:31.044231 12705 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.378s	user 0.001s	sys 0.000s
I20260709 12:10:31.037271 12659 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:31.117966 10581 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.007s	sys 0.004s
W20260709 12:10:31.118578 10581 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.008s	sys 0.004s
W20260709 12:10:31.104604 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76015.7us
W20260709 12:10:31.044884 12517 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.921s	user 0.012s	sys 0.001s
I20260709 12:10:31.108317 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.107380 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:31.020937 12679 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 12:10:30.995673 11215 rpcz_store.cc:276] 0709 12:10:29.732717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.734941 (+  2224us) service_pool.cc:224] Handling call
0709 12:10:30.969000 (+1234059us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400542ed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:30.969137 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149315328}
I20260709 12:10:31.096086 12636 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:31.143679 12362 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.111621 12524 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.108359 12657 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:31.148005 12721 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:10:31.148487 12703 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:31.123327 12740 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:31.150756 12725 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:10:31.025586 12595 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.155830 12711 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.157336 12743 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.159224 12735 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.159286 12587 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:31.126060 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:31.160408 12707 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 12:10:31.134846 12517 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.211170 12700 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.163147 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1630) took 1044 ms. Trace:
I20260709 12:10:31.212656 12700 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.211896 10430 rpcz_store.cc:276] 0709 12:10:30.120414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.120539 (+   125us) service_pool.cc:224] Handling call
0709 12:10:30.120891 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:30.133687 (+ 12796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.133710 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:30.134162 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:10:30.369127 (+234965us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.369186 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.369265 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.162847 (+793582us) raft_consensus.cc:1685] finished
0709 12:10:31.162872 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.163066 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13413,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":16753,"thread_start_us":184924,"threads_started":1,"wal-append.queue_time_us":334078}
I20260709 12:10:31.213197 12700 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.214371 10506 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:31.215193 12587 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.216199 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1372) took 1106 ms. Trace:
I20260709 12:10:31.216403 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:31.216598 10506 rpcz_store.cc:276] 0709 12:10:30.109372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.109543 (+   171us) service_pool.cc:224] Handling call
0709 12:10:30.130023 (+ 20480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:31.214813 (+1084790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.215898 (+  1085us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000da554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:31.216106 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493193600}
I20260709 12:10:31.217108 11221 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:31.217289 12587 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.217751 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 338) took 1048 ms. Trace:
I20260709 12:10:31.217496 12701 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.218096 11221 rpcz_store.cc:276] 0709 12:10:30.169830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.177755 (+  7925us) service_pool.cc:224] Handling call
0709 12:10:31.216209 (+1038454us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:31.217661 (+  1452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3414272}
W20260709 12:10:31.136831 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 136478us
W20260709 12:10:31.163599 10553 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.017s	sys 0.037s
I20260709 12:10:31.054272 12516 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:31.219842 10553 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.017s	sys 0.037s
I20260709 12:10:31.121412 12660 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:31.220247 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 663970us
I20260709 12:10:31.221400 12516 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:31.221506 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 95542.1us
I20260709 12:10:31.222275 12748 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.222905 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.222822 12748 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.223659 12517 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.224534 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.225512 12748 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.225708 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.188134 12656 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.122042 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:31.218143 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1631) took 1097 ms. Trace:
I20260709 12:10:31.227986 12656 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:31.227360 10436 rpcz_store.cc:276] 0709 12:10:30.120813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.120967 (+   154us) service_pool.cc:224] Handling call
0709 12:10:30.148009 (+ 27042us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:30.160072 (+ 12063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.160122 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:30.168603 (+  8481us) log.cc:844] Serialized 52 byte log entry
0709 12:10:30.497527 (+328924us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.658856 (+161329us) log.cc:844] Serialized 14 byte log entry
0709 12:10:30.659377 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.659495 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.159608 (+500113us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:31.217829 (+ 58221us) raft_consensus.cc:1685] finished
0709 12:10:31.217943 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.218080 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82610,"prepare.queue_time_us":440004,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3620,"raft.queue_time_us":12065,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":78111,"spinlock_wait_cycles":16392960,"thread_start_us":369332,"threads_started":2,"wal-append.queue_time_us":692638}
I20260709 12:10:31.144068 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:31.230707 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 86869.5us
I20260709 12:10:31.119983 12736 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.232260 12517 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.232662 12517 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:31.233004 12736 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.120577 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1624) took 1006 ms. Trace:
I20260709 12:10:31.235459 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1635) took 1099 ms. Trace:
I20260709 12:10:31.235221 12362 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:31.234680 10581 rpcz_store.cc:276] 0709 12:10:30.114528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.115466 (+   938us) service_pool.cc:224] Handling call
0709 12:10:30.129455 (+ 13989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.129728 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:30.364287 (+234559us) log.cc:844] Serialized 14 byte log entry
0709 12:10:30.533372 (+169085us) log.cc:844] Serialized 14 byte log entry
0709 12:10:31.119117 (+585745us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:31.119197 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:31.119271 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.119379 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.120496 (+  1117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":262160,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":2364,"raft.queue_time_us":48172,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":52290,"spinlock_wait_cycles":73031296,"thread_start_us":930562,"threads_started":3,"wal-append.queue_time_us":450382}
W20260709 12:10:31.149694 12710 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:10:31.106086 12669 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.235881 10434 rpcz_store.cc:276] 0709 12:10:30.136513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.136642 (+   129us) service_pool.cc:224] Handling call
0709 12:10:30.137015 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:30.183585 (+ 46570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.183635 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:30.184335 (+   700us) log.cc:844] Serialized 52 byte log entry
0709 12:10:30.492794 (+308459us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.535960 (+ 43166us) log.cc:844] Serialized 14 byte log entry
0709 12:10:30.536295 (+   335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.536379 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.227931 (+691552us) spinlock_profiling.cc:243] Waited 7.29 ms on lock 0x7b1800319960. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:31.235102 (+  7171us) raft_consensus.cc:1685] finished
0709 12:10:31.235218 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.235389 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31519,"prepare.queue_time_us":415476,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":101216,"raft.run_cpu_time_us":507,"raft.run_wall_time_us":79116,"spinlock_wait_cycles":16759680,"thread_start_us":261593,"threads_started":2,"wal-append.queue_time_us":348417}
I20260709 12:10:31.238489 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1633) took 1105 ms. Trace:
I20260709 12:10:31.145520 12356 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.179286 12657 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:31.239554 12356 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.240078 12356 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:31.240494 12356 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.238904 10437 rpcz_store.cc:276] 0709 12:10:30.133001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.133268 (+   267us) service_pool.cc:224] Handling call
0709 12:10:30.148586 (+ 15318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:30.389062 (+240476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.389130 (+    68us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:30.430962 (+ 41832us) log.cc:844] Serialized 52 byte log entry
0709 12:10:30.627008 (+196046us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.878669 (+251661us) log.cc:844] Serialized 14 byte log entry
0709 12:10:31.023264 (+144595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.023424 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.238251 (+214827us) raft_consensus.cc:1685] finished
0709 12:10:31.238277 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.238425 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":151668,"prepare.queue_time_us":327302,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":359698,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":19975,"thread_start_us":507478,"threads_started":4,"wal-append.queue_time_us":269092}
I20260709 12:10:31.119851 12734 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 12:10:31.241961 12734 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:31.242331 12734 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:31.243248 12734 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.244521 10504 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:31.244640 12734 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:31.246944 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.144899 12749 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.129s
I20260709 12:10:31.249887 12749 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:31.250281 12749 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:31.250373 12711 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.251322 12749 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.252473 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:31.255865 12655 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:31.200685 12743 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:31.258435 12743 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.259618 12669 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.260076 12711 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:31.123245 12731 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.281s
I20260709 12:10:31.264456 12731 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.264863 12731 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:31.265259 12731 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.269407 12719 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:31.272183 12598 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.190s	user 0.014s	sys 0.000s
I20260709 12:10:31.272737 12598 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.273945 11212 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:31.274279 12711 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:31.275995 12598 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.233023 12517 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.207844 12684 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.283530 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:31.284037 12684 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:31.284463 10436 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.284659 12684 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:31.243289 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 121349us
W20260709 12:10:31.286870 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67246.1us
I20260709 12:10:31.201742 12520 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.183990 12524 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:31.219489 12692 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:31.188301 12664 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.929s	user 0.016s	sys 0.006s
I20260709 12:10:31.178190 10511 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:31.308205 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:31.318336 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.318852 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:31.319788 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:31.319897 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:31.287806 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:31.336990 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.602s (SENT)
I20260709 12:10:31.140923 12508 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:31.357774 10438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.005s	sys 0.006s
W20260709 12:10:31.379347 10438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.005s	sys 0.006s
I20260709 12:10:31.360158 10434 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:31.383436 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 95725.6us
I20260709 12:10:31.384631 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:31.392354 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:31.393046 11178 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.725s (SENT)
I20260709 12:10:31.394116 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:31.221621 12660 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:31.291885 12735 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:31.398605 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 90567.3us
I20260709 12:10:31.373981 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1629) took 1250 ms. Trace:
I20260709 12:10:31.407788 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.373993 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:31.359587 10430 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.327916 12346 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.404752 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:31.288889 12268 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:31.406145 10587 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:31.407361 10586 rpcz_store.cc:276] 0709 12:10:30.123379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.215285 (+ 91906us) service_pool.cc:224] Handling call
0709 12:10:30.215605 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.275017 (+ 59412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.275072 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:30.275135 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.275187 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.373728 (+1098541us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b5400337054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:31.373789 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.373920 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135838464}
I20260709 12:10:31.372529 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.425982 10436 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:31.389040 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:31.442358 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48328.5us
I20260709 12:10:31.378997 12664 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.374648 12671 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.357890 12659 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:31.447083 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.031s
I20260709 12:10:31.374174 12688 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:31.355183 12548 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:10:31.376435 12749 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.396109 12598 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:31.461556 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.000s	sys 0.032s
I20260709 12:10:31.277489 12706 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.413556 12735 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.396727 12516 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:10:31.189932 12740 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.442564 12656 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:31.459771 12527 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.004s
I20260709 12:10:31.470888 12688 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.456890 12744 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:31.471473 12671 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.439960 12752 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:31.451128 12520 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:31.507074 12692 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:31.509177 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 283518us
I20260709 12:10:31.511951 10430 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:31.423846 12346 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.416875 12660 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.449472 12700 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:31.478616 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:31.507120 12715 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.556s	user 0.013s	sys 0.000s
I20260709 12:10:31.558686 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.500439 12744 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:31.509959 12520 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:31.509258 11215 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:31.517679 12664 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.507632 12268 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.578475 12679 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:31.573942 12520 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:31.474035 12659 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.588174 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.553346 12701 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:31.599726 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 178851us
I20260709 12:10:31.510056 12527 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:31.603091 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 266143us
I20260709 12:10:31.604143 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.571246 12744 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:31.587195 12656 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.621842 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.623674 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:31.619195 10553 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.532s (SENT)
W20260709 12:10:31.625749 10553 leader_election.cc:336] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.648s (SENT)
I20260709 12:10:31.626228 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.573532 12700 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
W20260709 12:10:31.630061 10553 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.729s (SENT)
I20260709 12:10:31.625859 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:31.607288 12670 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.008s
I20260709 12:10:31.638666 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.647446 10579 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:31.655531 12346 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:31.656143 12758 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:31.677994 12655 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.605132 12756 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:31.687412 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58453.6us
W20260709 12:10:31.688784 10553 leader_election.cc:336] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.740s (SENT)
I20260709 12:10:31.689371 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.625496 12587 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:31.679060 12655 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.608331 12753 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:31.695268 10553 leader_election.cc:336] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.002s (SENT)
W20260709 12:10:31.649690 12680 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.074s	user 0.013s	sys 0.004s
W20260709 12:10:31.680368 12715 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:10:31.700091 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 285668us
I20260709 12:10:31.630165 12656 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.718518 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.728598 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:31.729737 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:31.649906 12670 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.626524 12755 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 12:10:31.750020 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:31.753702 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1369) took 1652 ms. Trace:
I20260709 12:10:31.754249 10512 rpcz_store.cc:276] 0709 12:10:30.100881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.102006 (+  1125us) service_pool.cc:224] Handling call
0709 12:10:30.113127 (+ 11121us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:30.113423 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.113451 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:30.152065 (+ 38614us) log.cc:844] Serialized 26 byte log entry
0709 12:10:30.557084 (+405019us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:30.609542 (+ 52458us) log.cc:844] Serialized 14 byte log entry
0709 12:10:30.630762 (+ 21220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.630854 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.162475 (+531621us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b1800121a20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:31.506224 (+343749us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:31.753389 (+247165us) raft_consensus.cc:1685] finished
0709 12:10:31.753431 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.753613 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18893,"prepare.queue_time_us":54211,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":28232,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":14058,"spinlock_wait_cycles":13479168,"thread_start_us":364597,"threads_started":1,"wal-append.queue_time_us":712877}
I20260709 12:10:31.757436 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:31.726820 12753 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.706696 12680 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:31.773756 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 148008us
I20260709 12:10:31.778815 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:31.771484 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 147912us
I20260709 12:10:31.782763 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.786228 11219 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:31.761242 12719 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.795086 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:31.796768 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:31.797537 10578 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:31.772475 11216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 298) took 1153 ms. Trace:
I20260709 12:10:31.801371 10580 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:31.767251 12587 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:31.784498 12671 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:31.804203 12515 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.234s	user 0.008s	sys 0.008s
I20260709 12:10:31.799014 12688 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.835079 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1631) took 1711 ms. Trace:
I20260709 12:10:31.807521 12753 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.801076 12346 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:31.808384 12587 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:31.803601 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:31.835793 10582 rpcz_store.cc:276] 0709 12:10:30.123903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.318588 (+194685us) service_pool.cc:224] Handling call
0709 12:10:30.335760 (+ 17172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.373781 (+ 38021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.373817 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:30.373865 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.373897 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.834666 (+1460769us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54003dc054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:31.834744 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.834962 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86810368}
I20260709 12:10:31.837113 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.849999 10578 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:31.862349 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.812147 12660 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.818233 12659 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.824379 12719 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.852350 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:31.851312 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 121674us
I20260709 12:10:31.864903 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:31.868948 10583 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:31.818583 12740 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.869138 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:31.889475 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 102381us
I20260709 12:10:31.903923 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:31.906579 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:31.865453 12587 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:31.865139 12736 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.009s
I20260709 12:10:31.914040 12491 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:31.914595 12491 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:31.915061 12491 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.888432 12742 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:31.917791 12742 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:31.907095 12764 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:10:31.922482 12768 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:10:31.903105 12757 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:10:31.888440 12666 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.925942 12666 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.929009 12666 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.932107 12656 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.872481 12746 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:31.936348 12679 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:31.939469 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.939554 12670 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:31.890290 12659 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:31.940625 12739 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:31.941043 12659 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.952642 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1647) took 1191 ms. Trace:
I20260709 12:10:31.953504 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:31.954247 10582 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:31.913141 12778 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:31.854126 12706 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.955533 12778 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.863173 12527 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:31.855225 12753 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:31.957923 12778 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:31.961583 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55753.4us
I20260709 12:10:31.844125 12515 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:31.953119 10438 rpcz_store.cc:276] 0709 12:10:30.761579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.762210 (+   631us) service_pool.cc:224] Handling call
0709 12:10:30.778299 (+ 16089us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:31.380221 (+601922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:31.380258 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:31.380763 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:10:31.810157 (+429394us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:31.810209 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.810294 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.952377 (+142083us) raft_consensus.cc:1685] finished
0709 12:10:31.952402 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.952568 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":242134,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":22493,"thread_start_us":964986,"threads_started":2,"wal-append.queue_time_us":455417}
I20260709 12:10:31.863118 12346 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:31.949702 12697 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.992s	user 0.019s	sys 0.001s
I20260709 12:10:31.977614 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:31.980063 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:31.982524 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:31.983525 10436 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:31.985234 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:31.987334 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1640) took 1156 ms. Trace:
I20260709 12:10:31.987782 10584 rpcz_store.cc:276] 0709 12:10:30.831181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.833444 (+  2263us) service_pool.cc:224] Handling call
0709 12:10:30.879335 (+ 45891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.999932 (+120597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.987072 (+987140us) spinlock_profiling.cc:243] Waited 97.9 ms on lock 0x7b5400212d54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:31.987258 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225092992}
I20260709 12:10:31.958771 12753 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:31.965005 12736 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:32.008970 12744 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.015172 12779 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.968715 12670 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:31.973038 12706 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:31.991083 12778 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:31.805212 11216 rpcz_store.cc:276] 0709 12:10:30.568017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.704459 (+136442us) service_pool.cc:224] Handling call
0709 12:10:30.759058 (+ 54599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.820365 (+ 61307us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:31.011952 (+191587us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c8f9d50ac 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c964d73c7
0709 12:10:31.388075 (+376123us) log.cc:844] Serialized 14 byte log entry
0709 12:10:31.632420 (+244345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:31.640407 (+  7987us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:31.693185 (+ 52778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.703117 (+  9932us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.713695 (+ 10578us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5367,"prepare.queue_time_us":663994,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":72896,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":11147,"spinlock_wait_cycles":32976000,"thread_start_us":605978,"threads_started":2,"wal-append.queue_time_us":329812}
W20260709 12:10:32.029812 12356 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.789s	user 0.014s	sys 0.001s
I20260709 12:10:31.918787 12742 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:32.032866 12517 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.752s	user 0.006s	sys 0.007s
I20260709 12:10:32.033298 12517 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.035871 12346 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:32.036247 12749 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.001s
W20260709 12:10:32.038286 12657 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.799s	user 0.006s	sys 0.006s
I20260709 12:10:32.038748 12657 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.040325 12515 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:32.040899 12735 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.008s
I20260709 12:10:32.041308 12706 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:32.026486 12659 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:32.042744 12731 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.002s
I20260709 12:10:32.042441 12706 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:32.043577 12684 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.758s	user 0.014s	sys 0.000s
I20260709 12:10:32.044008 12684 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:32.024068 12698 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.912s	user 0.006s	sys 0.008s
I20260709 12:10:32.045573 12744 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.045485 12698 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.046319 12778 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:32.047181 12734 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.802s	user 0.008s	sys 0.007s
I20260709 12:10:32.047669 12698 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.047365 12778 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.048199 12670 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:32.041373 12735 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.049193 12670 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:32.049854 12743 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.004s
I20260709 12:10:32.050276 12743 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.051353 12735 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:32.051748 12714 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.104s	user 0.009s	sys 0.000s
I20260709 12:10:32.027619 12739 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:31.911172 12758 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:32.041122 12787 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:10:32.030392 12356 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:32.052366 12743 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.094130 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1370) took 1992 ms. Trace:
I20260709 12:10:32.149559 12587 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.043169 12731 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.148520 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.062472 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.155957 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.157277 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.158886 12524 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:32.159914 12524 consensus_queue.cc:1243] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:10:32.158746 12356 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:32.162590 12727 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.200s	user 0.003s	sys 0.008s
I20260709 12:10:32.163358 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.164209 10435 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:32.164608 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:32.165375 12709 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.220s	user 0.012s	sys 0.000s
I20260709 12:10:32.162829 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.169039 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:32.121506 12520 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:10:32.174556 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:32.166777 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1643) took 1970 ms. Trace:
I20260709 12:10:32.176107 10431 rpcz_store.cc:276] 0709 12:10:30.198929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.199105 (+   176us) service_pool.cc:224] Handling call
0709 12:10:30.199532 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:30.467732 (+268200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.468113 (+   381us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:30.494141 (+ 26028us) log.cc:844] Serialized 26 byte log entry
0709 12:10:30.774630 (+280489us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:31.063717 (+289087us) log.cc:844] Serialized 14 byte log entry
0709 12:10:31.065705 (+  1988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.065812 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:32.161894 (+1096082us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:32.166482 (+  4588us) raft_consensus.cc:1685] finished
0709 12:10:32.166589 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.166716 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125124,"prepare.queue_time_us":202038,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":212991,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":31791,"spinlock_wait_cycles":13903232,"thread_start_us":622707,"threads_started":3,"wal-append.queue_time_us":414479}
I20260709 12:10:32.178491 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.179620 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:32.181614 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:32.182205 10553 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.002s (SENT)
I20260709 12:10:32.182374 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.182709 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.183334 10584 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:10:32.183923 10553 leader_election.cc:336] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.032s (SENT)
I20260709 12:10:32.184368 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.184458 10584 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:32.186297 12659 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:32.186841 12659 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.187395 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.188187 10582 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:32.188828 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.189026 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.189282 12659 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.189814 12659 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:32.190178 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.190266 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.190506 12659 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:32.190975 10582 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:32.198388 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1638) took 2019 ms. Trace:
I20260709 12:10:32.203500 12701 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:32.171002 12520 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.204180 12701 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.205248 12701 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.210908 12741 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:32.211417 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.211645 12741 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:32.212857 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.092180 12744 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:32.212999 12741 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.214066 12744 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:32.215234 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 346162us
I20260709 12:10:32.229977 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.227546 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.240715 10437 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.198833 10580 rpcz_store.cc:276] 0709 12:10:30.178742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.574771 (+396029us) service_pool.cc:224] Handling call
0709 12:10:30.625616 (+ 50845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.807649 (+1182033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.198151 (+390502us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400336654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:32.198325 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2699187968}
I20260709 12:10:32.219090 12760 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:32.219900 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:32.220722 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:32.223106 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.224730 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.275952 10578 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:32.277303 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.278126 10578 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:32.227546 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.279428 10432 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:10:32.280789 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.282377 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.194450 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.284683 10581 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:32.286412 10581 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:32.198448 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.214587 12741 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.150147 10509 rpcz_store.cc:276] 0709 12:10:30.101164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.102071 (+   907us) service_pool.cc:224] Handling call
0709 12:10:30.123845 (+ 21774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:30.154406 (+ 30561us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:30.154445 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:30.202133 (+ 47688us) log.cc:844] Serialized 26 byte log entry
0709 12:10:30.495525 (+293392us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:30.495669 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.496900 (+  1231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:32.050914 (+1554014us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:32.059860 (+  8946us) raft_consensus.cc:1685] finished
0709 12:10:32.059897 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.094009 (+ 34112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8613,"raft.queue_time_us":25221,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7382,"spinlock_wait_cycles":13390208,"thread_start_us":194288,"threads_started":1,"wal-append.queue_time_us":470340}
I20260709 12:10:32.242556 10435 raft_consensus.cc:2410] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:32.292301 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.293618 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.294765 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.193672 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.296031 10584 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:32.239526 12517 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.296758 10432 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.236867 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.298029 10436 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5.
I20260709 12:10:32.299185 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.300171 10436 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:32.194088 10553 leader_election.cc:336] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.059s (SENT)
I20260709 12:10:32.300922 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:32.302383 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 108321us
I20260709 12:10:32.216815 12744 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.249260 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.304739 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.306465 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.248536 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.248632 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.262238 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.307597 10434 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:32.307940 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.206707 12520 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:32.247251 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.149550 12779 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:32.308470 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.301093 10583 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.309433 12779 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:32.309814 10434 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.310714 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.310550 12779 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.312204 12779 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.313333 10434 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:10:32.314972 10434 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:32.315575 12664 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:32.308799 10433 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.194914 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:32.317941 10433 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:10:32.317812 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 72422.2us
W20260709 12:10:32.320662 10553 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.234s (SENT)
I20260709 12:10:32.320859 12788 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:32.321249 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.321677 12788 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.324678 10431 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.295360 10435 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.324087 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.329408 10436 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:32.148633 12657 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:32.260267 12785 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:32.336164 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:32.336593 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:32.337606 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75460.4us
I20260709 12:10:32.339972 11177 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:32.267511 12717 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:32.352648 10578 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:32.327850 12664 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.352751 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.343727 11215 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:10:32.346976 12711 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.005s	sys 0.049s
I20260709 12:10:32.333292 12684 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.374689 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:32.386152 11219 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Advancing to term 6
I20260709 12:10:32.344966 12680 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.392398 10432 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:32.344676 12520 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:32.401479 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.394789 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.377604 10583 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:32.396535 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:32.374737 12775 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:32.376765 12711 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.006s	sys 0.049s
I20260709 12:10:32.354197 12661 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:32.354384 12671 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.419718 10513 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:10:32.422111 10512 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:32.423625 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.424713 10509 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:32.405452 11218 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:32.402496 10511 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:32.404793 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.429287 10508 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:32.404057 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51894.2us
I20260709 12:10:32.430691 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.387920 12760 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.394485 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:32.418759 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.431493 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.403358 10432 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.431849 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.444921 10513 raft_consensus.cc:2368] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6.
I20260709 12:10:32.447039 12752 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.447646 12752 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.348316 12785 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.379859 12664 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.450644 12731 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:32.447649 12801 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:10:32.355599 12717 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.452759 12664 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.453567 12788 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:32.456729 12718 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.497s	user 0.000s	sys 0.018s
I20260709 12:10:32.452818 12515 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.457896 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.458086 12515 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.459616 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.461163 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.461860 10509 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:32.456818 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.468241 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.470407 10510 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:32.450682 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1379) took 1609 ms. Trace:
I20260709 12:10:32.434592 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.415783 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.402073 12790 raft_consensus.cc:1027] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: attempt to promote peer 52bae2d2caf94d97b1049ecd4f82a2df: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:10:32.374511 12748 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 12:10:32.486449 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.486619 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.487892 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.519340 10510 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:32.471832 10505 rpcz_store.cc:276] 0709 12:10:30.840961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.841276 (+   315us) service_pool.cc:224] Handling call
0709 12:10:30.906582 (+ 65306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.018557 (+111975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.450408 (+1431851us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54000f8cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:32.450610 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255604608}
W20260709 12:10:32.439169 12491 consensus_meta.cc:287] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.524s	user 0.003s	sys 0.009s
I20260709 12:10:32.487924 12515 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.500411 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.522504 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.485383 10436 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:32.509845 10507 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:32.409158 12775 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.488435 12752 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.520808 10508 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:32.531129 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.436022 12733 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.532917 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.546131 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.547227 10512 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:32.526511 12784 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:10:32.495967 11216 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.552197 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.553113 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:32.554517 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:32.539042 12684 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.560241 10510 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.442229 12671 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.562422 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:32.374657 12758 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.562914 10505 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:32.559808 12719 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.533577 12775 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.579217 12663 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.579788 12663 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.580178 12663 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.544698 12752 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:32.571002 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.565129 12684 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.532364 12491 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.568653 12719 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.567112 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.526186 12697 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:10:32.544015 12733 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.584685 12548 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:32.597288 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.600936 10436 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.598716 11221 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:32.602212 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.603129 10432 raft_consensus.cc:2376] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4. Re-sending same reply.
I20260709 12:10:32.586135 12587 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:32.591863 12719 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:32.609246 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.629643 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.631266 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:32.633894 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52212.6us
I20260709 12:10:32.600538 12548 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:32.592507 12744 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:10:32.589587 12752 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.636706 10436 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.615440 12717 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.647783 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.650432 10512 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 2: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:32.655198 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.658815 10585 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:32.661072 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.665141 10436 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.638935 12548 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.671105 10435 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:32.673015 12684 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:32.674887 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.675285 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:32.675876 10435 raft_consensus.cc:2368] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
I20260709 12:10:32.676818 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.677386 10435 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:32.680048 12806 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:10:32.681041 12798 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:10:32.683439 12671 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.597113 12760 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.687945 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.688670 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:32.690747 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:32.694447 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.704715 10438 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.705363 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 347) took 2289 ms. Trace:
I20260709 12:10:32.705955 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.732092 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.702600 10431 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:32.708813 10435 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:32.750819 11219 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:32.757326 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.767325 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:32.757851 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.716116 12752 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.688341 12733 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.792068 10436 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:32.787124 10510 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.786793 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.753165 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.799574 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.804317 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:32.804910 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52149.4us
I20260709 12:10:32.805177 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.709914 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:32.782755 10509 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:32.791307 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:32.799577 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:32.808504 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53159.6us
I20260709 12:10:32.809468 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.810236 10512 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:32.800057 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.792236 12508 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.789018 12548 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.811652 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.812170 12514 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.812640 12514 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.813124 12548 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.808462 12697 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:32.711347 12789 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:32.707896 12681 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 12:10:32.816838 10436 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.780491 12671 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.811721 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.705811 11212 rpcz_store.cc:276] 0709 12:10:30.416180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.416370 (+   190us) service_pool.cc:224] Handling call
0709 12:10:30.416862 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.274362 (+857500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.705049 (+1430687us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b5400593154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:32.705262 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970634112}
I20260709 12:10:32.820101 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.820767 10578 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:32.840307 10438 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.834564 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.841594 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.821493 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.843253 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:32.844303 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.845778 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.812857 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.850294 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.851154 10505 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:32.852231 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:32.853152 10505 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:32.856739 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 206) took 1071 ms. Trace:
I20260709 12:10:32.857278 10586 rpcz_store.cc:276] 0709 12:10:31.785804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.800687 (+ 14883us) service_pool.cc:224] Handling call
0709 12:10:31.817050 (+ 16363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.016615 (+199565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.016650 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.016701 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:32.016742 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.856388 (+839646us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400208fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:32.856462 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.856641 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458340096}
I20260709 12:10:32.827443 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.859191 10512 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:32.860878 10512 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:10:32.861277 10512 pending_rounds.cc:85] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 4
I20260709 12:10:32.861544 10512 pending_rounds.cc:107] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:10:32.861835 10512 raft_consensus.cc:2889] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:32.865315 12734 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:10:32.866863 12723 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.907s	user 0.002s	sys 0.010s
I20260709 12:10:32.866957 12734 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.867770 12734 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.870514 12734 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.734274 12598 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:32.871182 12598 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:32.873023 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.872423 12598 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.875525 12770 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:32.876291 12770 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.877532 12770 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.884735 12751 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:32.887172 12789 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.888684 12789 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.891772 12739 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:32.892367 12739 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.893437 12739 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.895161 12697 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.896589 12697 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.819439 10432 raft_consensus.cc:2393] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:32.830394 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.842865 10578 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:32.837853 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.832818 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.903887 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:32.905267 10512 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:32.906805 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.784297 10403 leader_election.cc:400] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6.
I20260709 12:10:32.909946 10403 leader_election.cc:403] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:10:32.911372 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 127283us
I20260709 12:10:32.914338 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:32.923239 10584 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:32.915501 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.916425 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:32.932329 12739 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:32.917500 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.712980 12498 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:10:32.918061 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.830976 12680 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:32.922694 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.841396 10513 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:32.967310 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1637) took 2793 ms. Trace:
W20260709 12:10:32.830348 12706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.013s	sys 0.026s
W20260709 12:10:32.967922 12706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.014s	sys 0.027s
I20260709 12:10:32.967720 10587 rpcz_store.cc:276] 0709 12:10:30.173622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.470166 (+296544us) service_pool.cc:224] Handling call
0709 12:10:30.470535 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.415516 (+944981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.967013 (+1551497us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b540010b3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:32.967214 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151182720}
I20260709 12:10:32.731693 12683 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 12:10:32.969173 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.969441 12683 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:32.969871 12683 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:32.723098 12775 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.824391 12659 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.971124 12515 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:32.971369 12659 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.972822 12775 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.818472 12740 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:32.973454 12779 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:32.918074 12781 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:32.973960 11217 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:32.974084 12779 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:32.915999 12719 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.970750 12683 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.824651 12664 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:32.919548 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.977224 10578 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.915632 12671 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.938939 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:32.978550 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.981590 10583 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:32.982406 12706 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:32.940917 10581 raft_consensus.cc:2376] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5. Re-sending same reply.
I20260709 12:10:32.983287 10436 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:32.982999 12706 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:32.984977 10581 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:10:32.985407 10581 pending_rounds.cc:85] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 2
I20260709 12:10:32.985706 10581 pending_rounds.cc:107] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:32.986030 10581 raft_consensus.cc:2889] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:32.976678 12683 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.991220 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1658) took 1662 ms. Trace:
I20260709 12:10:32.991379 12659 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.991732 10429 rpcz_store.cc:276] 0709 12:10:31.328676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.328920 (+   244us) service_pool.cc:224] Handling call
0709 12:10:31.379862 (+ 50942us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:31.380437 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:31.380480 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:31.423204 (+ 42724us) log.cc:844] Serialized 52 byte log entry
0709 12:10:32.800572 (+1377368us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b1800411dc0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:32.940597 (+140025us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:32.956469 (+ 15872us) log.cc:844] Serialized 14 byte log entry
0709 12:10:32.956713 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.956795 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:32.990782 (+ 33987us) raft_consensus.cc:1685] finished
0709 12:10:32.990944 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.991131 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15252,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":10615,"raft.run_cpu_time_us":681,"raft.run_wall_time_us":75242,"spinlock_wait_cycles":2558848,"thread_start_us":139762,"threads_started":1,"wal-append.queue_time_us":1534783}
I20260709 12:10:32.994055 12671 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:32.935973 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.967969 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:32.978817 10585 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:32.996083 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 60282.9us
I20260709 12:10:32.997207 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:32.952468 10510 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:32.999876 10510 pending_rounds.cc:85] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 2
I20260709 12:10:33.000180 10510 pending_rounds.cc:107] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:33.000483 10510 raft_consensus.cc:2889] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:32.954501 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:32.958413 12520 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:10:32.959184 12680 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:32.832333 12668 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.131s
I20260709 12:10:32.839658 12299 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 12:10:32.840448 12346 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.131s
I20260709 12:10:32.964682 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:32.965157 12587 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:32.966367 12491 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:32.923341 12778 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6.
I20260709 12:10:32.985502 11216 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:33.003240 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:33.083763 10585 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.003939 10583 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:10:33.006434 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52054.4us
I20260709 12:10:32.922467 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:33.086623 10586 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:33.014094 10578 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:33.023975 10432 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:33.027741 12719 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:33.027521 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:33.039407 12818 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:33.039614 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 207386us
I20260709 12:10:32.976504 12664 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.080718 10582 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.026300 12788 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:33.079900 12760 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:33.096309 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.087841 12659 raft_consensus.cc:2764] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:10:33.019914 12671 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:10:33.093690 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:33.102635 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.164763 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:33.165868 10432 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:33.162868 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:33.180795 10478 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.506s (SENT)
I20260709 12:10:33.162971 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:33.195195 12664 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:33.196862 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:33.095630 12788 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:33.094971 12760 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:33.215941 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:33.103807 12778 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.097572 12785 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:33.235352 12814 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:10:33.215553 12762 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 1.442s	user 0.003s	sys 0.011s
W20260709 12:10:33.246955 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 242981us
I20260709 12:10:33.204900 12719 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:33.217941 12788 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:33.162683 12268 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
W20260709 12:10:33.261642 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 158173us
I20260709 12:10:33.097038 12779 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:33.269204 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:33.262939 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:33.275801 12760 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.277549 12760 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.258282 12778 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.280676 12749 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.281391 12749 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.256882 12706 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:33.283581 12663 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.008s
I20260709 12:10:33.295578 10585 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:33.284231 12663 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:33.318158 10506 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:33.233839 12514 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:33.321877 10504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1378) took 2480 ms (client timeout 2351 ms). Trace:
W20260709 12:10:33.322502 10504 rpcz_store.cc:269] 0709 12:10:30.833141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.833450 (+   309us) service_pool.cc:224] Handling call
0709 12:10:33.313271 (+2479821us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400413a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:33.313431 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925414784}
W20260709 12:10:33.297858 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 207737us
I20260709 12:10:33.303272 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 353) took 1087 ms. Trace:
I20260709 12:10:33.323907 11221 rpcz_store.cc:276] 0709 12:10:32.216323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.223814 (+  7491us) service_pool.cc:224] Handling call
0709 12:10:32.224228 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:32.608065 (+383837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:33.303000 (+694935us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54004bee54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:33.303204 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860602240}
I20260709 12:10:33.303686 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:33.331395 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:33.345314 10478 leader_election.cc:336] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.641s (SENT)
I20260709 12:10:33.346099 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:33.308513 12587 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.347565 12706 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.349539 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:33.350276 10505 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.339367 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:33.351923 10582 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.269110 12719 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:33.360795 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 97949.3us
I20260709 12:10:33.382699 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1636) took 3210 ms. Trace:
I20260709 12:10:33.389210 10553 leader_election.cc:400] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4.
W20260709 12:10:33.390908 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.383332 10579 rpcz_store.cc:276] 0709 12:10:30.173322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.465529 (+292207us) service_pool.cc:224] Handling call
0709 12:10:30.465978 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:31.681073 (+1215095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:33.382391 (+1701318us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400336154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:33.382608 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2707137280}
I20260709 12:10:33.352720 12680 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:33.366444 12824 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:33.327316 12765 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:10:33.348434 12706 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:33.433348 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 53838.8us
I20260709 12:10:33.435098 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:33.437573 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54628.5us
I20260709 12:10:33.384449 12818 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:33.438912 10582 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:33.440608 12818 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.441007 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:33.442710 12665 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:33.443522 12665 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:33.465246 12733 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.466300 12733 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:33.467350 12671 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:33.471510 12811 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:33.472404 12811 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:33.476749 12809 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:33.477331 12809 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:33.392426 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:33.479452 12506 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:33.480129 12506 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:33.481490 12506 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.444880 12665 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:33.484414 12826 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:10:33.488596 12514 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:33.491730 10507 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:33.494817 12733 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:33.496061 12663 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:33.497413 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.414353 12656 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:33.514564 10505 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:33.424927 12680 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.498724 12719 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:33.414048 12790 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 12:10:33.425161 12830 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:33.522112 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:33.525630 10507 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:33.525868 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 133552us
I20260709 12:10:33.523537 12829 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:33.539187 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:33.478528 12809 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:33.522943 12781 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:33.542721 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:33.533910 12671 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:33.533133 12506 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.524955 12824 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:33.551846 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.540294 12830 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:33.557930 12681 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.559105 12829 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:33.563710 12671 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.598838 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:33.596570 12671 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.602039 10431 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:33.604943 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:33.605841 10433 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:33.608665 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.613143 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:33.614902 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:33.616092 10436 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:33.620554 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:33.621519 10433 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.584340 12809 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:33.629567 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:33.630300 10507 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
W20260709 12:10:33.631610 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 362473us
I20260709 12:10:33.632710 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:33.634359 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:33.636082 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 199867us
I20260709 12:10:33.636823 10429 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:33.612632 12739 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.674s	user 0.004s	sys 0.011s
I20260709 12:10:33.636643 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:33.640790 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:33.641760 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.575220 12781 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:33.657828 10436 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:33.658212 10478 leader_election.cc:336] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.935s (SENT)
I20260709 12:10:33.658718 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:33.612146 12811 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:33.662689 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:33.663918 10429 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.635550 12742 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:33.592041 12681 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:33.641194 12761 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:10:33.674099 10505 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:33.651170 12498 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:33.666153 12683 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.677s	user 0.015s	sys 0.000s
I20260709 12:10:33.640548 12587 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:33.668174 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:33.655567 12775 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:33.656347 12268 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.680608 12706 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:33.678793 12824 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.683569 10436 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:33.684790 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:33.685691 10436 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:33.691560 12829 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.693511 12790 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:33.694060 12790 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:33.665325 12657 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.656462 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.701435 10433 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:33.691720 12498 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:33.711179 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.713102 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:33.714807 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58468us
I20260709 12:10:33.715595 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.716908 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:33.721312 10552 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:33.671715 12779 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:33.809587 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92783.6us
I20260709 12:10:33.813828 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:33.816102 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:33.714078 12680 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.718863 12706 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.843233 10433 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:33.838442 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:33.719848 12775 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:33.812561 12779 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:33.823357 12587 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.828699 12758 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.710796 12819 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 12:10:33.880164 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:33.861519 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 147540us
I20260709 12:10:33.845062 12778 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.850127 12818 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.862964 12680 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:33.861677 12739 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:33.845310 12842 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:33.878021 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.876652 12779 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.894347 12506 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:33.875041 12683 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:10:33.883066 12761 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:10:33.697393 12830 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:33.911744 12788 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.624s	user 0.015s	sys 0.000s
I20260709 12:10:33.883544 12684 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:10:33.905184 12739 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.916567 12810 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:33.975993 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:33.988358 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 265586us
I20260709 12:10:33.908182 12657 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:10:33.916746 12776 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:33.989507 10552 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:33.711236 12790 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:33.918361 12843 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 12:10:33.936693 12506 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:33.986526 12739 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.980774 12788 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.013672 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:33.916560 12498 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.002672 12681 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:34.022903 12684 consensus_queue.cc:1243] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:33.934664 12775 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.017829 12524 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.991866 12810 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.047924 10429 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:34.051290 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.691s (SENT) [suppressed 29 similar messages]
I20260709 12:10:34.028245 12843 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:34.108901 12706 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.111255 12854 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:34.112794 12841 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:10:34.115221 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:34.044852 12790 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:34.117833 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 301804us
I20260709 12:10:34.043642 12842 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:34.107652 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56411.8us
I20260709 12:10:34.024238 12778 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:10:34.115239 12847 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.105949 12840 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:34.107028 12850 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:10:34.137471 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 123941us
I20260709 12:10:34.014283 12776 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.026700 12852 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 12:10:34.097950 12524 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.135746 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:34.168103 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:34.168984 12524 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.169322 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:34.169605 12524 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:34.128237 12842 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:34.171591 12844 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 12:10:34.145789 12778 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:34.147641 12508 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:34.148137 12587 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:34.125360 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:34.157475 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:34.158392 10553 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.549s (SENT)
I20260709 12:10:34.179870 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.180595 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.182004 10552 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:34.164383 12836 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:10:34.130350 12706 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.165206 12742 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:34.187649 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:34.187088 12852 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:34.190398 12788 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.126821 12506 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:34.191061 12839 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 12:10:34.179342 12587 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:34.129783 12854 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.206516 11216 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:34.156445 12847 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.179751 12508 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:34.188616 12819 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:34.217485 11221 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:34.213068 10430 raft_consensus.cc:3055] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:34.218679 10430 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:34.219031 12853 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:10:34.219894 10430 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.220315 12857 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:34.221418 10430 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:34.188853 12850 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:34.220521 12810 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.215952 12838 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:34.216573 11217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 302) took 2861 ms (client timeout 1867 ms). Trace:
W20260709 12:10:34.227110 12846 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 12:10:34.227473 10437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 169) took 2163 ms (client timeout 2037 ms). Trace:
I20260709 12:10:34.228163 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:34.228991 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.229664 12595 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:34.231721 12859 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:34.233186 12860 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:34.234524 12860 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:34.172144 12842 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.234963 12587 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.236467 12842 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:34.227985 10437 rpcz_store.cc:269] 0709 12:10:32.061886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.062085 (+   199us) service_pool.cc:224] Handling call
0709 12:10:34.224584 (+2162499us) spinlock_profiling.cc:243] Waited 82.2 ms on lock 0x7b5400435b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:34.224732 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189142912}
I20260709 12:10:34.209229 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:34.254119 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44948.6us
I20260709 12:10:34.212260 12852 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:34.256532 10552 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:34.236883 12587 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:34.269691 10430 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:10:34.226354 11217 rpcz_store.cc:269] 0709 12:10:31.355220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.357365 (+  2145us) service_pool.cc:224] Handling call
0709 12:10:32.675102 (+1317737us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:34.216142 (+1541040us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b540051fed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:34.216391 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703256192}
I20260709 12:10:34.268865 12850 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:34.293294 12850 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:34.281472 10552 connection.cc:256] RPC call timeout handler was delayed by -1.310550160000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2351s for a response
I20260709 12:10:34.294013 12850 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:34.294318 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.300518 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.300778 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.300275 12852 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.303048 10578 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:34.284310 12756 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:10:34.257846 12859 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:34.308965 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.293440 10582 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:34.310657 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.311348 10584 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:34.305531 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.305985 12749 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.310686 12859 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:34.308264 10585 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:34.314349 12859 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:34.277211 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 164) took 2289 ms. Trace:
I20260709 12:10:34.306699 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.315101 10430 rpcz_store.cc:276] 0709 12:10:31.981578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.981720 (+   142us) service_pool.cc:224] Handling call
0709 12:10:32.004556 (+ 22836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:34.212957 (+2208401us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54002e3220. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae11c0 00007f2c92ae2231 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c
0709 12:10:34.270254 (+ 57297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.270553 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359240960}
I20260709 12:10:34.318496 10578 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:34.328125 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:34.328572 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.326475 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.330508 10578 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:34.333218 10585 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:34.333418 10584 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:34.333784 10585 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:34.334887 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 106866us
I20260709 12:10:34.335455 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.334851 10585 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.337400 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.338114 10585 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:34.340276 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.345527 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.346258 10581 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:34.359234 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.360280 10583 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:34.262560 12857 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:10:34.315258 12742 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:10:34.365420 10585 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:34.372113 10578 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:34.366694 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:34.372845 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 202423us
I20260709 12:10:34.346716 12854 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:34.372810 12346 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.285s	user 0.010s	sys 0.008s
I20260709 12:10:34.383900 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:34.387049 12346 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.011s	sys 0.008s
I20260709 12:10:34.393963 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:34.311546 12508 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.396066 10478 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:34.382555 12809 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.002s
I20260709 12:10:34.399770 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:34.377133 12670 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:34.403900 12858 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:34.403935 12681 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:34.380921 12683 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.406945 12683 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.410022 12524 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:34.410175 12683 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.412321 12681 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:34.414167 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.415226 12514 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:34.415759 12514 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:34.416227 12514 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:34.416468 12684 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:34.418048 12684 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:34.420825 11219 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:34.418582 12684 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:34.392313 12752 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:34.389940 12832 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:10:34.425490 11214 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:34.425285 12671 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.790s	user 0.011s	sys 0.005s
I20260709 12:10:34.419934 12657 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:34.462101 12268 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.751s	user 0.013s	sys 0.001s
I20260709 12:10:34.428146 12524 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:34.462494 12508 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:34.392540 12733 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:10:34.515249 12346 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:34.515489 12809 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:34.484567 12671 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.522681 12514 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.524978 12857 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.491798 12268 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:34.529237 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.424381 12670 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.538846 12524 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:34.423508 12776 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:34.561861 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71979.1us
I20260709 12:10:34.565187 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.568264 12514 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.568570 12671 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:34.571803 12670 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.572829 12683 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:34.575919 12524 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.576923 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.578362 12752 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:34.579257 12719 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:34.580039 12851 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:10:34.582544 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:34.583767 12514 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:34.584450 12514 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:34.585104 12514 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:34.586297 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.590561 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:34.595793 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.596499 10583 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:34.597366 12870 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:34.605615 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.608157 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.604665 12683 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:10:34.608817 10582 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:34.543620 12809 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.610574 12362 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.611119 12362 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.543368 12346 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.593472 12752 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:34.603268 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.598204 12870 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.719113 10584 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:10:34.719694 10584 pending_rounds.cc:85] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 3
I20260709 12:10:34.719982 10584 pending_rounds.cc:107] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:10:34.720268 10584 raft_consensus.cc:2889] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:34.607151 12779 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.005s
I20260709 12:10:34.552274 12857 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.706353 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1918) took 2218 ms. Trace:
I20260709 12:10:34.707661 12697 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.425486 12858 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.129s
W20260709 12:10:34.713230 12790 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.006s
I20260709 12:10:34.747872 10512 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:34.713594 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.749135 10512 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:34.714253 12781 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:34.714681 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.541961 12847 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.737926 10582 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:34.765779 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.739173 10513 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:34.742616 10506 rpcz_store.cc:276] 0709 12:10:32.400171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.400593 (+   422us) service_pool.cc:224] Handling call
0709 12:10:33.313582 (+912989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:33.328943 (+ 15361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.618681 (+1289738us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b5400413a50. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:34.618870 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057618944}
I20260709 12:10:34.744562 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.503108 12850 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.769186 10507 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:34.549559 12684 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.772413 10511 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:34.770422 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.779143 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:34.779804 10435 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
W20260709 12:10:34.780360 10435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1347) took 2048 ms (client timeout 2692 ms). Trace:
W20260709 12:10:34.780750 10435 rpcz_store.cc:269] 0709 12:10:32.731646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.778818 (+2047172us) service_pool.cc:224] Handling call
0709 12:10:34.780301 (+  1483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:34.770840 12809 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:34.749804 12779 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.768956 12838 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
W20260709 12:10:34.793648 12739 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.769s	user 0.011s	sys 0.005s
I20260709 12:10:34.761659 12697 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.795032 12346 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:34.794781 12684 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:34.786329 12659 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:10:34.767752 12829 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:10:34.827610 12733 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:34.828328 12733 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.831167 12850 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:34.844128 12872 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.305s	user 0.001s	sys 0.001s
I20260709 12:10:34.869691 12514 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:34.867380 12868 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 12:10:34.883141 10504 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:34.886860 12857 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:34.887059 12514 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.887928 12870 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:34.888803 12514 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.819940 12739 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.891089 12697 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:34.892725 12779 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.893193 12870 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.901291 10508 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:34.902021 12760 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:34.908337 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.910519 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:34.911156 10581 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:34.917234 10512 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:34.909355 12809 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.921326 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.925565 10510 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:34.923568 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.923949 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:34.908162 12760 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:34.926270 10504 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:34.912170 12781 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.942729 10581 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:34.907649 12514 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.945729 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.947494 10581 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:34.920583 12779 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:10:34.922099 12520 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:10:34.911245 12870 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:34.947404 12810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.016s	sys 0.006s
I20260709 12:10:34.965173 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:34.932138 12809 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:34.941900 12781 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:34.967399 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:34.974562 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.972342 12739 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.986965 12781 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:34.987342 10586 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:34.987761 10586 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:34.988555 10586 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.989944 10586 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:34.991155 12659 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:34.991710 12659 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.993888 12659 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:34.995129 12840 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.995674 12840 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:34.997473 10585 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:34.997673 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:34.997961 10585 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:34.998852 10430 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:34.999796 12514 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.000320 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:35.001829 10436 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:35.000833 10431 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:35.001780 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:35.003413 10429 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:34.997809 12840 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.004060 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:35.004460 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:35.004648 10430 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:35.006732 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:34.961376 12760 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.007704 10434 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:35.008075 10434 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:35.008483 12760 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.008828 10434 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:35.009919 11178 connection.cc:256] RPC call timeout handler was delayed by -1.1475845069999195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
I20260709 12:10:35.010046 10434 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.013535 11219 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:10:35.016228 12829 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:35.017143 12829 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:35.017463 12829 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:35.018281 12829 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.027762 10508 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:34.971500 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:35.023787 10585 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.033281 10585 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.035962 11216 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:35.036310 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1692) took 1428 ms. Trace:
I20260709 12:10:35.036736 10432 rpcz_store.cc:276] 0709 12:10:33.611120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.645897 (+ 34777us) service_pool.cc:224] Handling call
0709 12:10:33.667159 (+ 21262us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:33.677291 (+ 10132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:33.677342 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:33.695645 (+ 18303us) log.cc:844] Serialized 52 byte log entry
0709 12:10:33.984193 (+288548us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:33.984267 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:33.984377 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:34.978514 (+994137us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b2400172b98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa30fb0 00007f2c926a7488 00007f2c926a7be8 00007f2c9267f398 00007f2c9267a76a 00007f2c92679bb6
0709 12:10:35.036018 (+ 57504us) raft_consensus.cc:1685] finished
0709 12:10:35.036040 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:35.036246 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6106,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":20678,"spinlock_wait_cycles":7996160,"thread_start_us":152339,"threads_started":1,"wal-append.queue_time_us":206053}
I20260709 12:10:35.036356 12752 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:10:35.040365 12362 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.064393 11212 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:35.067313 12739 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:35.069358 12739 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.066511 10434 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:10:35.070312 12827 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:10:35.071357 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.071619 12514 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:35.073232 12514 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.073618 12514 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:35.073989 12514 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.069914 12739 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.075369 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.076335 11216 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:35.071537 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 183) took 1978 ms. Trace:
I20260709 12:10:35.082005 10586 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:10:35.082409 10586 pending_rounds.cc:85] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 2
I20260709 12:10:35.082626 10586 pending_rounds.cc:107] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:35.082856 10586 raft_consensus.cc:2889] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:35.086300 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:35.087771 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.089596 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 356) took 1045 ms. Trace:
I20260709 12:10:35.089701 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.090013 11221 rpcz_store.cc:276] 0709 12:10:34.044248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.044394 (+   146us) service_pool.cc:224] Handling call
0709 12:10:34.217270 (+172876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:34.217913 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.085932 (+868019us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540051fed0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:35.089521 (+  3589us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272976256}
I20260709 12:10:35.090492 10586 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:35.099998 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.105299 10586 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:35.091823 12299 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:10:35.079571 12508 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.103112 10430 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:35.109611 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.104471 10585 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:10:35.100416 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.108296 11212 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:10:35.089777 12829 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:34.968041 12810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.017s	sys 0.007s
I20260709 12:10:35.112191 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:35.113256 10585 pending_rounds.cc:85] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 6
I20260709 12:10:35.113627 10585 pending_rounds.cc:107] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:10:35.113934 10585 raft_consensus.cc:2889] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:35.078874 10434 rpcz_store.cc:276] 0709 12:10:33.093319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.093456 (+   137us) service_pool.cc:224] Handling call
0709 12:10:33.178952 (+ 85496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:35.007599 (+1828647us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001cda20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae11c0 00007f2c92ae2231 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c
0709 12:10:35.070076 (+ 62477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.071472 (+  1396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263851776}
I20260709 12:10:35.111383 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.120102 12740 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:10:35.122192 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.122743 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.125875 12881 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:10:35.117056 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.131742 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.133996 10585 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:35.099125 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:35.150805 10585 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:35.133129 11212 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 2.
W20260709 12:10:35.148298 12859 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.833s	user 0.008s	sys 0.009s
I20260709 12:10:35.162453 12859 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.164037 10587 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:35.159690 11212 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:10:35.168481 12882 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:10:35.168841 12859 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:35.177035 12749 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.177670 12749 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.180178 12749 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.182003 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.185699 12809 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:35.098996 12756 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:10:35.160844 12870 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.148538 12779 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.195935 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.196724 10511 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:35.152513 12668 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:35.115576 12508 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.158237 11216 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:35.205027 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.207643 11212 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:35.220157 12668 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:35.223124 12668 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.226630 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.227686 10583 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:35.228569 12756 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:35.229575 12756 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.230479 12852 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:35.231173 12852 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.231873 12756 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.234227 12852 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.234794 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.266732 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.204960 12779 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.274420 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.283089 10583 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:35.284751 12779 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.274529 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:35.223872 12508 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.299549 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.300246 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:35.290098 12870 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.296921 12811 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.307341 12684 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.319375 12684 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:35.319329 12811 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.319981 12684 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.322101 12811 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:35.322518 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 106) took 10913 ms. Trace:
I20260709 12:10:35.288272 12659 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:10:32.006969  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:35.310762 12749 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:35.311038 12514 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.299239 12508 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:35.326629 10581 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:35.327792 11219 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:35.311798 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:35.335019 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.335476 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:35.314348 11216 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:35.346556 12749 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 12:10:35.353084 12514 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.352878 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 357) took 1308 ms. Trace:
I20260709 12:10:35.371419 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.377754 10507 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: [3.5-5.7]
I20260709 12:10:35.379063 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:35.381723 10508 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.379393 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:35.383056 10513 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:35.384074 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.388092 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.378952 10478 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.389994 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.383955 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.392007 10507 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:35.392551 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.393106 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:35.393838 10507 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.395372 10512 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:35.346827 12508 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:35.372918 12664 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:35.346796 12858 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.322907 10336 rpcz_store.cc:276] 0709 12:10:24.436271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.436800 (+   529us) service_pool.cc:224] Handling call
0709 12:10:35.322424 (+10885624us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:28.281972 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:28.282762 (+   790us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:28.282829 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:10:28.282870 (+    41us) tablet.cc:662] Decoding operations
0709 12:10:28.284251 (+  1381us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:28.284361 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:28.284395 (+    34us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:10:28.303822 (+ 19427us) tablet.cc:701] Row locks acquired
0709 12:10:28.303882 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:10:28.304145 (+   263us) write_op.cc:270] Start()
0709 12:10:28.304327 (+   182us) write_op.cc:276] Timestamp: P: 1783599028304006 usec, L: 0
0709 12:10:28.304350 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:10:28.305108 (+   758us) log.cc:844] Serialized 4107 byte log entry
0709 12:10:29.273378 (+968270us) op_driver.cc:464] REPLICATION: finished
0709 12:10:29.581183 (+307805us) write_op.cc:301] APPLY: starting
0709 12:10:29.586928 (+  5745us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:29.587300 (+   372us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:29.587478 (+   178us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:29.597361 (+  9883us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:29.597753 (+   392us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:10:29.597826 (+    73us) write_op.cc:312] APPLY: finished
0709 12:10:29.598280 (+   454us) log.cc:844] Serialized 77 byte log entry
0709 12:10:29.598512 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:29.598675 (+   163us) write_op.cc:454] Released schema lock
0709 12:10:29.598760 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:29.656534 (+ 57774us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:29.659114 (+  2580us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2104,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":271041,"apply.run_cpu_time_us":5171,"apply.run_wall_time_us":66002,"num_ops":10,"prepare.queue_time_us":196281,"prepare.run_cpu_time_us":20092,"prepare.run_wall_time_us":151128,"raft.queue_time_us":723033,"raft.run_cpu_time_us":5197,"raft.run_wall_time_us":148330,"replication_time_us":968714,"thread_start_us":851284,"threads_started":4,"wal-append.queue_time_us":157188,"wal-append.run_cpu_time_us":143272,"wal-append.run_wall_time_us":2150332}]]}
I20260709 12:10:35.414172 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.422959 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.423764 10511 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:35.387506 12827 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.412237 12514 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:35.412983 12508 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:35.429857 11217 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:35.416086 12664 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:35.436399 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.412240 12760 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:35.441943 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:35.419971 12595 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:35.420840 12858 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:35.445014 10508 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:35.456588 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.462060 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:35.462899 10507 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:35.444439 12752 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:35.377815 11214 rpcz_store.cc:276] 0709 12:10:34.044732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.047087 (+  2355us) service_pool.cc:224] Handling call
0709 12:10:34.111478 (+ 64391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:34.471255 (+359777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.352565 (+881310us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54004074d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:35.352800 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663976704}
I20260709 12:10:35.466431 12664 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.467937 12664 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:35.460255 12858 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.471391 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:35.472087 10506 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:35.474280 10511 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:35.475329 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:35.475644 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:35.476583 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:35.476817 12810 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.477273 10434 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:35.478296 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:35.478960 10430 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:35.444638 12827 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.459264 12514 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:35.480701 12595 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:35.481303 10438 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:35.445989 12508 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:35.464718 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.454501 12760 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:35.480980 12514 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.429527 12748 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:10:35.484503 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.486253 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.476627 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:35.482070 12659 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.485555 12752 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:35.459652 12851 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:35.493307 12684 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.481544 12870 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:35.497593 12659 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:35.503748 10479 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.507206 12659 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.507282 12775 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.467952 12706 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.482664 12830 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:35.512735 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:35.538462 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:35.518682 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.546432 10578 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:35.498204 12851 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:35.520529 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 242196us
I20260709 12:10:35.501274 12756 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:35.521720 10430 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:35.522043 12668 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:35.522853 12520 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:35.523111 12661 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:35.481774 12595 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.525050 12739 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:35.525738 10505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 173) took 1983 ms (client timeout 2201 ms). Trace:
I20260709 12:10:35.495457 12760 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.555015 12362 raft_consensus.cc:2764] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:10:35.560551 12683 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:35.527686 12752 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.528532 12733 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.532509 12598 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.551007 12668 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:35.562816 12668 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:35.544641 10506 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:35.567358 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:35.567720 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.568063 11214 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:35.551229 10586 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:35.552254 10511 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:35.487900 12779 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 12:10:35.528053 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.555584 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:35.555367 10505 rpcz_store.cc:269] 0709 12:10:33.542164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.542368 (+   204us) service_pool.cc:224] Handling call
0709 12:10:35.525537 (+1983169us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002c65d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:35.525668 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260762368}
I20260709 12:10:35.571381 10430 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:35.572947 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 361) took 1008 ms. Trace:
I20260709 12:10:35.573402 11217 rpcz_store.cc:276] 0709 12:10:34.564249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.566765 (+  2516us) service_pool.cc:224] Handling call
0709 12:10:34.567401 (+   636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:35.457464 (+890063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.572644 (+115180us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b5400593154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:35.572872 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942848896}
I20260709 12:10:35.574865 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:35.575538 10431 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.577440 12717 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.557747 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:35.512429 12775 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:35.578037 12717 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.579030 10435 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:35.579274 10581 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:35.579381 11212 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:35.581750 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1672) took 1273 ms. Trace:
I20260709 12:10:35.582144 10587 rpcz_store.cc:276] 0709 12:10:34.307912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.308674 (+   762us) service_pool.cc:224] Handling call
0709 12:10:35.581516 (+1272842us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b540017a554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:35.581648 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964318592}
I20260709 12:10:35.550571 12851 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.578418 12775 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:35.583107 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.588502 10431 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:35.543139 12862 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:35.598685 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.600800 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:35.617882 10584 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:35.621763 10584 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:35.550697 12661 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:35.550627 12756 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:35.644743 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 230622us
W20260709 12:10:35.646445 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53174.2us
I20260709 12:10:35.650432 10584 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:35.684818 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:35.696540 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48914.7us
I20260709 12:10:35.616312 12862 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:35.624275 12740 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.694394 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:35.615623 12733 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.768404 12733 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:35.770700 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:35.645720 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:35.705865 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.709656 12866 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:35.714689 12840 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:35.716081 10333 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:35.683337 12827 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.724762 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:35.809350 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 84688.4us
I20260709 12:10:35.744408 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.811714 12697 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:35.812322 12697 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:35.812388 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.814038 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.766785 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:35.816394 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.817885 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.818807 10508 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:35.771507 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.819814 12534 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:35.820886 10583 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:35.812907 12697 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.822245 10583 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:35.823346 12656 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:35.823910 12656 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:35.825156 12656 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.772225 10585 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:35.772624 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.828958 10583 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:35.776403 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.684001 12756 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:35.833627 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.836416 12664 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.837157 12664 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.699342 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.782047 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.840714 12664 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:35.836436 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:35.847179 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69957.6us
W20260709 12:10:35.776737 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 131122us
W20260709 12:10:35.699766 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.848374 12656 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.814805 10507 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:35.773197 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:35.831269 12790 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.850736 10580 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 2.
I20260709 12:10:35.850777 12790 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.852639 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.853430 12790 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:35.854266 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.810426 10552 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:35.888942 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78654.6us
I20260709 12:10:35.889444 10581 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:35.890092 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:35.887799 10511 raft_consensus.cc:1217] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 12:10:35.764216 12862 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:35.920830 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:35.934942 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 438639us
W20260709 12:10:35.936226 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46287.4us
W20260709 12:10:35.936693 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.943125 10511 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:10:35.848369 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:35.945942 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 95779.6us
I20260709 12:10:35.918852 10507 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:35.919901 10333 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:35.888016 12830 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:35.793113 12866 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:35.770840 12733 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:35.776558 12740 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:35.927593 12788 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.855645 12840 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:35.930397 12810 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:35.919306 12664 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:35.938751 12827 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.937894 12668 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:35.977012 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 128771us
I20260709 12:10:35.955936 12748 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:35.979070 10512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.028s	sys 0.026s
I20260709 12:10:35.986867 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.987809 10333 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:10:35.995044 10512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.029s	sys 0.026s
I20260709 12:10:35.995599 10333 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:35.991051 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:35.979694 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.891721 12508 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.001132 11219 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:35.962795 12740 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.010473 10333 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:35.987929 12899 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:10:35.972357 12717 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:36.028169 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:36.028255 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48651.6us
I20260709 12:10:36.029989 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:35.960500 12534 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:10:35.994145 12733 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:35.923519 12661 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:35.999747 12684 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:36.055265 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:35.973964 12788 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.047602 10513 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 1. Current term is 3. Ops: []
I20260709 12:10:36.026615 12779 consensus_queue.cc:1243] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:10:36.051041 12858 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 12:10:35.999976 12667 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:10:36.065893 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:36.050174 12853 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:10:36.079423 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:36.082091 10431 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:36.085999 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56083us
I20260709 12:10:36.051942 12733 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.024628 12760 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.091953 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:36.072538 10333 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 3 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:36.000077 12827 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:36.100081 10435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.010s	sys 0.007s
I20260709 12:10:36.098740 10333 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:36.025426 12756 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.079041 12781 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:36.072613 12859 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:10:36.079747 12904 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:36.095343 12681 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:36.111200 10506 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:36.093536 12733 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.084054 12902 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:36.045930 12840 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.129956 10434 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:36.105931 12906 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.115741 10333 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:36.049724 12866 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.136457 10504 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:36.088846 12858 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:36.117647 12900 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:10:36.118069 12902 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.161677 10333 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:10:36.140388 12659 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
I20260709 12:10:36.114045 12904 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.160159 12866 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:36.154896 12840 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.172515 10509 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:36.109548 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 264216us
I20260709 12:10:36.163921 12902 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.138362 12906 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.184518 12904 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:36.267163 11176 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.017s	sys 0.022s
W20260709 12:10:36.269006 11176 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.018s	sys 0.023s
I20260709 12:10:36.227525 12668 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:36.275234 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.232106 12830 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:36.180258 12659 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.242944 12911 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:10:36.271847 12595 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.021s	sys 0.025s
W20260709 12:10:36.244896 12903 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:10:36.239751 12917 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.243405 12508 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:36.296876 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1695) took 1483 ms. Trace:
I20260709 12:10:36.297612 10586 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:36.297422 10582 rpcz_store.cc:276] 0709 12:10:34.813559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.813712 (+   153us) service_pool.cc:224] Handling call
0709 12:10:34.814105 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:34.814481 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:34.814503 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:34.814966 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:10:35.349368 (+534402us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b1800068980. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:35.537930 (+188562us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:35.585399 (+ 47469us) log.cc:844] Serialized 14 byte log entry
0709 12:10:35.585629 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.585704 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.296488 (+710784us) raft_consensus.cc:1685] finished
0709 12:10:36.296615 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.296793 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17986,"prepare.queue_time_us":51810,"raft.queue_time_us":5162,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":4523,"spinlock_wait_cycles":18695680,"thread_start_us":188340,"threads_started":1,"wal-append.queue_time_us":865035}
W20260709 12:10:36.290398 12595 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.022s	sys 0.025s
I20260709 12:10:36.292204 12788 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:36.301961 10333 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 4, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:36.327165 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:36.301192 12680 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:36.313678 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 247893us
I20260709 12:10:36.302521 12760 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:36.305075 12748 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:36.363755 10437 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:36.367528 10437 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:36.364748 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:36.369817 10429 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:36.369392 10333 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:10:36.314126 12851 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.000s
I20260709 12:10:36.368436 10437 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.379233 10437 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:36.383786 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.313069 12508 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.312067 12788 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:36.402257 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:36.350462 12781 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:36.390590 10429 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:36.368891 12928 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.372906 12595 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:36.395143 12905 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:10:36.332111 12756 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:36.412678 10479 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:36.295992 12917 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:36.417686 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 319977us
I20260709 12:10:36.394968 12858 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:36.425695 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.424767 10333 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 4, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:36.394728 12926 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:36.431788 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71823.9us
I20260709 12:10:36.440723 10333 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 2, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:36.417510 12595 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.446012 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:36.395624 12680 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.399897 12851 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.448498 10333 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:36.422561 12827 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:36.449934 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:36.407176 12508 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.369202 12830 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.416206 12788 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:36.475513 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.470829 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.473026 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.454391 11221 raft_consensus.cc:2376] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4. Re-sending same reply.
W20260709 12:10:36.481775 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.478199 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:36.468331 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:36.492929 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.477571 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:36.494735 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:36.452346 10333 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 3 to 5, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:36.451965 12595 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:36.496023 12852 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.500679 12937 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.500996 12948 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:36.501667 12915 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:10:36.502694 12740 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.502961 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1420) took 1120 ms. Trace:
W20260709 12:10:36.477990 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.423720 12928 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.503515 12941 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.467955 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:36.512004 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:36.560371 10581 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
W20260709 12:10:36.466614 12921 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:10:36.467584 12916 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:10:36.515416 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:36.575704 10553 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.580s (SENT)
I20260709 12:10:36.576046 10579 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
W20260709 12:10:36.516914 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.499286 10333 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:36.467135 12659 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:36.522604 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47384.2us
W20260709 12:10:36.504051 12775 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.883s	user 0.005s	sys 0.008s
I20260709 12:10:36.473598 12859 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.458115 12934 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:36.466882 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:36.547590 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:36.545111 10512 rpcz_store.cc:276] 0709 12:10:35.382255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.396141 (+ 13886us) service_pool.cc:224] Handling call
0709 12:10:35.396540 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:35.427209 (+ 30669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:35.427269 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:35.433562 (+  6293us) log.cc:844] Serialized 26 byte log entry
0709 12:10:35.995674 (+562112us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:35.995722 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.995803 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.502677 (+506874us) raft_consensus.cc:1685] finished
0709 12:10:36.502706 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.502883 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32095,"raft.queue_time_us":30902,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9996,"thread_start_us":523997,"threads_started":1,"wal-append.queue_time_us":648993}
I20260709 12:10:36.551766 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:36.556207 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47005.1us
I20260709 12:10:36.506186 12917 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:36.509775 12508 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:36.507256 12706 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:36.566064 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 77606.5us
I20260709 12:10:36.514295 12756 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:36.543015 12919 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:36.468125 12920 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:10:36.463955 12750 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 12:10:36.508095 12760 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:10:36.443964 12926 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:36.594691 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1735) took 1010 ms. Trace:
I20260709 12:10:36.595981 10584 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:36.464148 12748 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.594406 10333 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:36.557421 12362 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:36.571667 12697 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.750s	user 0.005s	sys 0.009s
I20260709 12:10:36.522782 12852 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.503551 12781 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.549397 12937 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:36.548285 12948 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:36.788797 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 209300us
I20260709 12:10:36.609983 11220 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:10:36.780433 10553 leader_election.cc:336] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.691s (SENT)
W20260709 12:10:36.663843 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.747092 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:36.795670 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:36.791126 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1722) took 1306 ms. Trace:
I20260709 12:10:36.789566 10435 rpcz_store.cc:276] 0709 12:10:35.584789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.584939 (+   150us) service_pool.cc:224] Handling call
0709 12:10:35.585248 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:35.585591 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:35.585618 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:35.589922 (+  4304us) log.cc:844] Serialized 26 byte log entry
0709 12:10:36.101162 (+511240us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:36.101211 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.101287 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.594407 (+493120us) raft_consensus.cc:1685] finished
0709 12:10:36.594432 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.594609 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":273939,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":34955,"thread_start_us":510877,"threads_started":1,"wal-append.queue_time_us":675232}
I20260709 12:10:36.792675 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:36.814532 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:36.792737 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:36.816924 10584 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:36.611687 12859 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.823050 10582 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:36.817656 10432 rpcz_store.cc:276] 0709 12:10:35.484845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.502640 (+ 17795us) service_pool.cc:224] Handling call
0709 12:10:35.538653 (+ 36013us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:35.539174 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:35.539204 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:35.554202 (+ 14998us) log.cc:844] Serialized 574 byte log entry
0709 12:10:36.088140 (+533938us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:36.088198 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.088305 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.610421 (+522116us) raft_consensus.cc:1685] finished
0709 12:10:36.610446 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.791046 (+180600us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":523,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":10313,"thread_start_us":462920,"threads_started":1,"wal-append.queue_time_us":686534}
W20260709 12:10:36.749545 12862 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.817s	user 0.014s	sys 0.000s
W20260709 12:10:36.738904 12656 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.890s	user 0.015s	sys 0.002s
I20260709 12:10:36.852905 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:36.861755 10583 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:36.866097 12859 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:36.633658 11217 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.671240 12934 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:36.786890 12950 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:36.787060 12927 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:36.588413 12827 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.274s
I20260709 12:10:36.793931 12829 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:36.891937 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 275002us
I20260709 12:10:36.786561 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:36.863565 10333 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:36.848551 12949 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:36.953816 10435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:36.971216 10435 raft_consensus.cc:606] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:36.970855 12949 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.971652 10435 raft_consensus.cc:614] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:36.871750 12862 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.973412 12949 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:36.974020 12949 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.611052 12940 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.975310 12940 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.669003 12917 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.214s
I20260709 12:10:36.674734 12706 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.585175 12924 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.325s
I20260709 12:10:36.595633 12775 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.923169 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:36.884318 12914 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.439s	user 0.000s	sys 0.002s
I20260709 12:10:36.596184 12947 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.338s
I20260709 12:10:36.859968 12851 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:36.891649 12859 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:37.015667 10333 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 4, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 4 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:37.021245 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 421204us
I20260709 12:10:37.018626 10333 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 3, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:37.036358 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.914616 12927 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.974557 12949 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.903198 12656 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.813537 12362 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.056828 10505 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:36.577744 12941 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:37.059280 12937 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:36.908123 12950 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.060123 12941 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:36.823902 12697 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:37.063818 12943 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:10:36.794507 12933 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.124s
I20260709 12:10:36.951691 10583 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:36.958848 12958 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 12:10:37.065115 12933 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:37.065516 12933 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:36.968214 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 139034us
I20260709 12:10:36.918540 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:36.927453 12829 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.815675 12919 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:36.977885 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1939) took 1433 ms. Trace:
W20260709 12:10:36.992465 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 172780us
I20260709 12:10:36.981228 12740 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:36.820757 12756 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:36.982401 12788 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.077092 10511 rpcz_store.cc:276] 0709 12:10:35.544897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.569764 (+ 24867us) service_pool.cc:224] Handling call
0709 12:10:35.775509 (+205745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:35.943767 (+168258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.977633 (+1033866us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540036d150. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:36.977813 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472321408}
I20260709 12:10:37.053653 11218 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:10:37.054062 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 181949us
I20260709 12:10:36.988924 12680 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:36.956964 10508 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:37.040369 12961 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.041025 12659 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:37.001645 12942 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
I20260709 12:10:37.021977 12934 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.051040 12917 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:37.051899 12827 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:37.269246 12914 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:10:37.286728 12948 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.066547 12852 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:37.092001 12902 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.819s	user 0.014s	sys 0.000s
I20260709 12:10:37.191704 12758 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:37.217099 12904 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.935s	user 0.014s	sys 0.000s
W20260709 12:10:37.269135 12840 consensus_meta.cc:287] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.036s	user 0.012s	sys 0.002s
I20260709 12:10:37.080860 12945 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.268771 12919 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.110489 12928 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.286460 12917 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:37.190362 12906 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.269748 12926 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.288897 12961 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.302790 12961 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.303059 12949 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.300168 12945 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:37.342031 10437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.021s	sys 0.020s
W20260709 12:10:37.343385 10437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.023s	sys 0.020s
W20260709 12:10:36.987515 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.349707 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 426395us
I20260709 12:10:37.352034 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1725) took 1840 ms. Trace:
I20260709 12:10:37.353497 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:37.359972 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.364601 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.366658 11212 raft_consensus.cc:1217] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Deduplicated request from leader. Original: 1.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 12:10:37.090808 11216 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:37.367836 11212 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:10:37.368680 12659 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.370123 12362 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.235250 12706 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.352690 10438 rpcz_store.cc:276] 0709 12:10:35.511684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.528354 (+ 16670us) service_pool.cc:224] Handling call
0709 12:10:35.528754 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:35.529469 (+   715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:35.529501 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:35.530158 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:10:36.024084 (+493926us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:36.308344 (+284260us) log.cc:844] Serialized 14 byte log entry
0709 12:10:36.370836 (+ 62492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.370946 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:37.351328 (+980382us) raft_consensus.cc:1685] finished
0709 12:10:37.351376 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.351794 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69734,"prepare.queue_time_us":109552,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":2780,"raft.queue_time_us":15697,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":21839,"thread_start_us":691347,"threads_started":2,"wal-append.queue_time_us":760287}
I20260709 12:10:37.372334 12775 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.067924 12656 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.375809 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:37.378175 12656 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.379506 12706 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:37.379861 12810 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.380113 12706 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:37.380820 12706 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:37.381047 12810 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.080148 12697 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:37.383865 12664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.358s	user 0.014s	sys 0.009s
W20260709 12:10:37.384487 12664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.361s	user 0.014s	sys 0.009s
I20260709 12:10:37.385113 12664 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:37.385277 12810 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.211887 12862 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:37.387456 12866 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.159s	user 0.012s	sys 0.005s
I20260709 12:10:37.294365 12902 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.389734 12756 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.294075 12758 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:37.390839 10512 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:37.392619 12902 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.392758 12758 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.393728 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:37.297268 12904 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.401809 12917 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.390113 12664 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.066597 12933 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.407981 12933 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.408643 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:37.414108 12733 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.280s	user 0.013s	sys 0.007s
I20260709 12:10:37.414964 12949 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.416572 12949 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.419495 12733 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.428099 12945 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.415313 12904 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.433511 12917 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:37.445794 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 187) took 1083 ms. Trace:
W20260709 12:10:37.468570 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 60189.2us
I20260709 12:10:37.470270 12733 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.487350 10510 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:37.473887 10511 rpcz_store.cc:276] 0709 12:10:36.312475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.352359 (+1039884us) service_pool.cc:224] Handling call
0709 12:10:37.395424 (+ 43065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:37.499794 12866 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:37.501271 12942 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:37.501942 10511 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:37.504134 10512 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:37.504947 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1430) took 1564 ms. Trace:
I20260709 12:10:37.505338 10512 rpcz_store.cc:276] 0709 12:10:35.940265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.501779 (+1561514us) service_pool.cc:224] Handling call
0709 12:10:37.502391 (+   612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.504627 (+  2236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.504890 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:37.506071 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 361) took 1064 ms. Trace:
I20260709 12:10:37.498883 12949 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.450897 12758 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.507982 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1429) took 1780 ms. Trace:
I20260709 12:10:37.512398 10511 rpcz_store.cc:276] 0709 12:10:35.797407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.500543 (+1703136us) service_pool.cc:224] Handling call
0709 12:10:37.501551 (+  1008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.502674 (+  1123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.502943 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:37.495848 12706 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.506551 11214 rpcz_store.cc:276] 0709 12:10:36.441366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.442750 (+  1384us) service_pool.cc:224] Handling call
0709 12:10:36.589057 (+146307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:36.893307 (+304250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:36.903346 (+ 10039us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:37.370626 (+467280us) log.cc:844] Serialized 574 byte log entry
0709 12:10:37.398392 (+ 27766us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:37.398486 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.398662 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:37.505509 (+106847us) raft_consensus.cc:1685] finished
0709 12:10:37.505537 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.505770 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31383,"raft.queue_time_us":203558,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":48816,"thread_start_us":26821,"threads_started":1,"wal-append.queue_time_us":27512}
I20260709 12:10:37.516849 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:37.518050 12940 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.519215 12942 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:37.522141 12950 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.536369 10511 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:37.537557 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 188) took 1092 ms. Trace:
I20260709 12:10:37.538121 10511 rpcz_store.cc:276] 0709 12:10:36.445189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.528932 (+1083743us) service_pool.cc:224] Handling call
0709 12:10:37.535910 (+  6978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.537132 (+  1222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.537440 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:37.543795 12706 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.084627 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:37.518817 12940 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.547621 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:37.547910 12940 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.515142 12756 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:37.558358 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1937) took 2014 ms. Trace:
I20260709 12:10:37.558462 12950 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.559587 12950 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.559127 10506 rpcz_store.cc:276] 0709 12:10:35.539419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.568654 (+ 29235us) service_pool.cc:224] Handling call
0709 12:10:35.569014 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.115637 (+546623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.550257 (+1434620us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b5400228dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.553643 (+  3386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246339968}
I20260709 12:10:37.562865 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1432) took 1047 ms. Trace:
I20260709 12:10:37.564594 10506 rpcz_store.cc:276] 0709 12:10:36.515569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.561342 (+1045773us) service_pool.cc:224] Handling call
0709 12:10:37.561917 (+   575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.562399 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.562432 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:37.562497 (+    65us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:37.562563 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.562632 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.562767 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:37.562760 12950 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.568588 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:37.571442 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1421) took 2179 ms. Trace:
I20260709 12:10:37.574249 10513 rpcz_store.cc:276] 0709 12:10:35.385808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.416199 (+ 30391us) service_pool.cc:224] Handling call
0709 12:10:35.416541 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.057746 (+641205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.563059 (+1505313us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b540039f8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.565387 (+  2328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450660608}
W20260709 12:10:37.587241 11219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 365) took 2102 ms (client timeout 1818 ms). Trace:
W20260709 12:10:37.587913 11219 rpcz_store.cc:269] 0709 12:10:35.484880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.485915 (+  1035us) service_pool.cc:224] Handling call
0709 12:10:37.586758 (+2100843us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b5400821c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:37.586960 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161444096}
I20260709 12:10:37.590448 12935 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:37.607304 11220 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:10:37.607882 11220 pending_rounds.cc:85] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 3
I20260709 12:10:37.608192 11220 pending_rounds.cc:107] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260709 12:10:37.608614 11220 raft_consensus.cc:2917] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260709 12:10:37.613803 11220 raft_consensus.cc:2889] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:37.604879 12756 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:37.623334 10553 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.445s (SENT)
I20260709 12:10:37.624547 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1428) took 1901 ms. Trace:
I20260709 12:10:37.625128 10510 rpcz_store.cc:276] 0709 12:10:35.722732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.126214 (+403482us) service_pool.cc:224] Handling call
0709 12:10:36.138700 (+ 12486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:37.488131 (+1349431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.624090 (+135959us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540038f9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.624464 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3064282880}
I20260709 12:10:37.629568 12661 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:37.632911 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 107850us
W20260709 12:10:37.639858 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.645825 10478 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:37.647248 12664 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.647475 12935 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:37.647929 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71212.3us
I20260709 12:10:37.648244 12935 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:37.648180 12664 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.648991 12935 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.649021 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:37.650100 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.650533 12858 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:37.651192 12949 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.651708 12680 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.652164 12949 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.652601 12680 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.655501 12680 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.655692 12949 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.664235 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:37.665005 12935 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.665345 12664 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.665428 12870 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:37.670264 12927 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.674620 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1711) took 1128 ms. Trace:
I20260709 12:10:37.675909 12756 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.682521 12756 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.681460 12870 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.691557 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1434) took 1108 ms. Trace:
I20260709 12:10:37.692065 10506 rpcz_store.cc:276] 0709 12:10:36.581905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.568392 (+986487us) service_pool.cc:224] Handling call
0709 12:10:37.600994 (+ 32602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.635553 (+ 34559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.635603 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:37.678064 (+ 42461us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:37.688262 (+ 10198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.689784 (+  1522us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.689877 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:37.699349 12936 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.699543 12733 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.700548 12969 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.700150 12936 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.700740 12668 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:37.701287 12969 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.701514 12668 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.702186 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1942) took 2103 ms. Trace:
I20260709 12:10:37.702620 10504 rpcz_store.cc:276] 0709 12:10:35.598216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.074885 (+476669us) service_pool.cc:224] Handling call
0709 12:10:36.096170 (+ 21285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.137519 (+ 41349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.701676 (+1564157us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b5400413f54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.701963 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92045696}
I20260709 12:10:37.703192 12902 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.703632 12969 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.704093 12668 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.703991 12902 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.705121 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1407) took 2995 ms. Trace:
I20260709 12:10:37.705603 10509 rpcz_store.cc:276] 0709 12:10:34.709596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.745462 (+ 35866us) service_pool.cc:224] Handling call
0709 12:10:35.527089 (+781627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.219840 (+692751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.584290 (+1364450us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54002c65d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.704639 (+120349us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54002c65d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:37.704882 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3280536704}
I20260709 12:10:37.706303 12902 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.709272 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:37.710335 11215 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:37.694458 12917 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:37.712545 11218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 365) took 1250 ms. Trace:
I20260709 12:10:37.698571 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:37.700416 12733 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.713331 11218 rpcz_store.cc:276] 0709 12:10:36.462136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.468046 (+  5910us) service_pool.cc:224] Handling call
0709 12:10:37.711889 (+1243843us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54006b1f54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:37.712205 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1281807744}
I20260709 12:10:37.718230 12756 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:37.700436 12534 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.723685 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:37.721494 12534 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.735397 12534 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.679529 10587 rpcz_store.cc:276] 0709 12:10:36.545902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.546050 (+   148us) service_pool.cc:224] Handling call
0709 12:10:36.546505 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.860652 (+314147us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:36.908189 (+ 47537us) log.cc:844] Serialized 14 byte log entry
0709 12:10:37.077663 (+169474us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:37.077761 (+    98us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:37.077830 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.674044 (+596214us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400276fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.674112 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.674368 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54206,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":1328,"raft.queue_time_us":38880,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":18744,"spinlock_wait_cycles":721771904,"thread_start_us":168973,"threads_started":1,"wal-append.queue_time_us":399053}
I20260709 12:10:37.731184 12917 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.739367 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:37.740270 10581 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:37.743614 12870 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:37.744285 12990 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 12:10:37.744541 11217 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.800s	user 0.010s	sys 0.014s
I20260709 12:10:37.745146 11217 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:37.746341 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48026.1us
W20260709 12:10:37.746599 11217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 360) took 1305 ms (client timeout 1501 ms). Trace:
W20260709 12:10:37.747398 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.747447 10508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 1950) took 2046 ms (client timeout 2223 ms). Trace:
I20260709 12:10:37.747570 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:37.732075 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 546839us
W20260709 12:10:37.749493 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58590.1us
I20260709 12:10:37.752070 12962 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.752636 10581 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:37.753575 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.754449 12534 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:37.754326 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.755543 12936 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:37.756028 12595 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.169s	user 0.008s	sys 0.015s
I20260709 12:10:37.718807 12940 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:37.757366 10508 rpcz_store.cc:269] 0709 12:10:35.700659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.836196 (+135537us) service_pool.cc:224] Handling call
0709 12:10:37.746886 (+1910690us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400412654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:37.747135 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370306304}
I20260709 12:10:37.755312 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:37.767712 10581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1720) took 1208 ms (client timeout 1597 ms). Trace:
I20260709 12:10:37.766593 12758 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:37.766709 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.771469 10513 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:37.759506 12534 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:37.768162 10581 rpcz_store.cc:269] 0709 12:10:36.562283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.584440 (+ 22157us) service_pool.cc:224] Handling call
0709 12:10:37.738965 (+1154525us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:37.764797 (+ 25832us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3839232}
W20260709 12:10:37.756551 11217 rpcz_store.cc:269] 0709 12:10:36.440829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.442625 (+  1796us) service_pool.cc:224] Handling call
0709 12:10:37.746497 (+1303872us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:37.779244 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.781405 10510 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:37.775425 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.784140 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:37.785055 10587 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.786388 10507 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:37.786899 10581 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:37.787369 10581 pending_rounds.cc:85] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 1
I20260709 12:10:37.765862 12595 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.787618 10581 pending_rounds.cc:107] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:37.787846 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.789449 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.789690 12595 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.790540 10507 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:10:37.787945 10581 raft_consensus.cc:2889] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:37.790735 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:37.774936 12534 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.791173 12733 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.791515 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:37.793644 12758 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:37.807430 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.782104 12962 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.823139 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.807842 12935 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.823932 10507 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:10:37.826301 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78923.9us
I20260709 12:10:37.819370 12733 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:37.832273 12733 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:37.832399 10510 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:37.833451 12733 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.830168 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:37.826124 12758 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:37.828227 10438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:37.839628 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1736) took 1765 ms. Trace:
I20260709 12:10:37.841976 10438 raft_consensus.cc:606] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:37.843293 10438 raft_consensus.cc:614] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:37.842247 10434 rpcz_store.cc:276] 0709 12:10:36.056043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.059966 (+  3923us) service_pool.cc:224] Handling call
0709 12:10:36.090005 (+ 30039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.133255 (+ 43250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.808930 (+1675675us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54003d7cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.820911 (+ 11981us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91245952}
I20260709 12:10:37.847648 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.864171 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.869840 12299 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:10:37.876274 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.877424 10507 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:37.878472 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.879936 10508 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:37.881109 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.882331 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.889266 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:37.890902 10581 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:37.891309 10513 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:37.879933 12962 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.896950 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:37.897437 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1718) took 1341 ms. Trace:
W20260709 12:10:37.901301 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.909600 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:37.910574 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:37.911299 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:37.911870 11215 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:37.912261 12758 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:37.912529 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 366) took 1450 ms. Trace:
I20260709 12:10:37.912801 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:37.912947 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 65308.4us
I20260709 12:10:37.914373 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 1940) took 2365 ms. Trace:
I20260709 12:10:37.914778 10505 rpcz_store.cc:276] 0709 12:10:35.548448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.571709 (+ 23261us) service_pool.cc:224] Handling call
0709 12:10:35.571977 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.057551 (+1485574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.913824 (+856273us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400413054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.914151 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3414218112}
I20260709 12:10:37.916782 11212 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:37.917279 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:37.903752 10585 rpcz_store.cc:276] 0709 12:10:36.555685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.076306 (+520621us) service_pool.cc:224] Handling call
0709 12:10:37.076775 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:37.354385 (+277610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.354447 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:37.355496 (+  1049us) log.cc:844] Serialized 26 byte log entry
0709 12:10:37.511911 (+156415us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:37.511956 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.896805 (+384849us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54002774d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.896862 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:37.896916 (+    54us) raft_consensus.cc:1685] finished
0709 12:10:37.896932 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.897218 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42561,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":22774,"spinlock_wait_cycles":635002752,"thread_start_us":154104,"threads_started":1,"wal-append.queue_time_us":183621}
W20260709 12:10:37.919845 10553 leader_election.cc:336] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.692s (SENT)
I20260709 12:10:37.920480 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:37.921870 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:37.923588 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:37.924233 10579 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:10:37.924464 11218 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:37.926625 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 1987) took 1432 ms. Trace:
I20260709 12:10:37.927089 10582 rpcz_store.cc:276] 0709 12:10:36.494092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.494449 (+   357us) service_pool.cc:224] Handling call
0709 12:10:37.926182 (+1431733us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54003895d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:37.926371 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752517888}
I20260709 12:10:37.914790 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:37.948019 10587 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:37.934894 12829 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:37.957659 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:37.952466 12962 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.976959 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:37.912962 11215 rpcz_store.cc:276] 0709 12:10:36.462530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.477832 (+ 15302us) service_pool.cc:224] Handling call
0709 12:10:37.708838 (+1231006us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bae9c 00007f2c8f9baa5b 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:37.912458 (+203620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39887104}
I20260709 12:10:37.981206 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 1969) took 2378 ms. Trace:
I20260709 12:10:37.981772 10586 rpcz_store.cc:276] 0709 12:10:35.599399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.599645 (+   246us) service_pool.cc:224] Handling call
0709 12:10:35.600031 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:36.298136 (+698105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.974179 (+1676043us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54001caf54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:37.978051 (+  3872us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603797888}
I20260709 12:10:37.984198 12830 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:37.984542 12667 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:37.985303 12830 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:37.985707 12830 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:37.990047 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:37.990232 12946 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:37.967896 12758 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:37.992017 12756 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:37.986744 12830 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:37.978116 12299 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:10:38.000694 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.005303 10435 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 6. Ops: []
I20260709 12:10:38.005182 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.006029 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:37.991392 12706 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:10:38.016925 10437 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:38.017350 12706 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:38.017805 12706 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:38.018806 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:38.019626 10584 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.019110 12706 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.020417 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:37.995553 12758 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.021595 11219 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:38.021687 12758 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:38.023350 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:38.014731 12993 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.385s	user 0.005s	sys 0.000s
I20260709 12:10:38.007680 12756 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:38.008266 12942 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:37.981299 12928 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:38.008649 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.012528 10432 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:38.013520 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.032871 10433 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:38.032946 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:38.034246 10429 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:38.007524 12830 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.037110 12829 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:38.037832 12829 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.027793 10509 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:38.040732 12829 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.042325 10581 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:38.043394 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.045039 10584 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.045660 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1721) took 1483 ms. Trace:
I20260709 12:10:38.045120 12756 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.046458 12927 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:38.047079 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.047305 12927 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.047818 10437 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.047333 12928 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.048808 10434 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:38.049778 12927 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.049917 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.051673 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.050717 10437 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.041304 12299 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:37.994451 12917 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.046020 10584 rpcz_store.cc:276] 0709 12:10:36.562670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.018181 (+1455511us) service_pool.cc:224] Handling call
0709 12:10:38.045600 (+ 27419us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:38.050493 12927 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.062038 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:37.993050 12962 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.062513 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.059458 10438 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:38.050606 12928 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.068077 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:38.072935 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:38.073671 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44666.1us
I20260709 12:10:38.079262 10434 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:10:38.081432 10433 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:38.079761 10434 pending_rounds.cc:85] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 5
W20260709 12:10:38.085388 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.085765 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.062794 12659 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:38.087162 10434 pending_rounds.cc:107] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:10:38.088824 10434 raft_consensus.cc:2889] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:38.089190 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.082055 10432 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.086462 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.079376 12706 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.083528 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.066221 12534 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.098055 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.098711 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:38.098871 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.085927 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.108677 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 189) took 1661 ms. Trace:
I20260709 12:10:38.099702 11212 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:38.098121 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.069262 12927 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.081013 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.085676 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.111822 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.110296 10432 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:38.109128 10511 rpcz_store.cc:276] 0709 12:10:36.446086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.543752 (+1097666us) service_pool.cc:224] Handling call
0709 12:10:37.544314 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.544769 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:37.553700 (+  8931us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.013885 (+460185us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:38.100925 (+ 87040us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b18002cb4e0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c969b358f 00007f2c8f9f2571 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd
0709 12:10:38.101416 (+   491us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.101622 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:38.101679 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:38.106969 (+  5290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.107092 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.107215 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":491137,"prepare.queue_time_us":490796,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":842,"raft.queue_time_us":3016,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":6459,"spinlock_wait_cycles":13345024,"thread_start_us":44929,"threads_started":1,"wal-append.queue_time_us":65460}
I20260709 12:10:38.115787 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.116772 10506 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.117830 12809 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:38.118402 12809 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:38.118675 12940 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:38.119606 12809 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.120137 12661 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:38.125200 10431 raft_consensus.cc:2410] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:38.125242 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.126504 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.126526 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.132717 10504 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:38.136685 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 186) took 1825 ms. Trace:
I20260709 12:10:38.123607 12917 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:38.142369 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.123404 12661 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:38.152206 10429 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.155408 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.151466 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.156322 10436 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.162688 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:38.164186 13002 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:10:38.164808 12299 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:10:38.165314 12299 raft_consensus.cc:1068] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:38.165994 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.165880 12299 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:38.166528 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1717) took 1611 ms. Trace:
I20260709 12:10:38.167092 12664 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.167433 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.167526 12664 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.168298 12664 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.167441 10580 rpcz_store.cc:276] 0709 12:10:36.555356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.929651 (+1374295us) service_pool.cc:224] Handling call
0709 12:10:37.930161 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:37.930816 (+   655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.930850 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:37.934485 (+  3635us) log.cc:844] Serialized 26 byte log entry
0709 12:10:38.028284 (+ 93799us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:38.028336 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.028440 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.166163 (+137723us) raft_consensus.cc:1685] finished
0709 12:10:38.166184 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.166331 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":93337,"threads_started":1,"wal-append.queue_time_us":136991}
I20260709 12:10:38.168900 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:38.170089 12949 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.170562 12949 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.170943 12949 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.172413 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:38.173247 12680 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:38.174823 12680 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:38.170140 12928 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:38.175575 12928 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:38.175957 12928 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:38.176690 10585 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 3: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.176961 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.177987 12299 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:38.178575 12299 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:38.179688 12299 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.182081 12809 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.185135 12680 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:38.186607 12680 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:38.187258 12680 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:38.175120 12934 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:38.188272 12829 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:38.188776 12829 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:38.188565 12934 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.189851 12829 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.192718 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.196489 12927 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:38.198246 12962 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.199882 12534 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:38.202446 12991 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:10:38.202703 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.203722 10578 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:38.123308 11213 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:38.165928 12661 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.159902 11216 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:38.207616 12661 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.207676 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.209052 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.210530 10438 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.210930 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.211493 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.212322 10434 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:38.212514 12514 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:38.213356 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.214178 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.214833 10436 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:38.215723 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.216778 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.217792 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.219707 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.221024 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.221766 10511 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:38.223052 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.223333 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.223740 10585 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.223999 10431 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:38.227458 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.228199 10585 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
I20260709 12:10:38.229442 12758 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.230361 12829 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.230919 10438 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:38.230978 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.232674 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:38.232760 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.241289 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.241946 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:38.242252 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.238642 10438 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:38.218971 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.244958 10436 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:38.209780 12934 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.240182 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.246503 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.253712 10578 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.214206 10434 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:38.238829 10404 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:38.250753 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.251106 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.267724 10438 raft_consensus.cc:2410] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:38.267518 10579 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:10:38.281895 10432 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:38.283806 12933 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.283965 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:38.284415 12933 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.284915 10505 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.284811 12933 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.287320 10429 raft_consensus.cc:2368] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 3. Current term is 4.
W20260709 12:10:38.288436 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.288328 12928 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.291839 10580 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:38.288509 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.295040 10583 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:38.296327 10587 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.296512 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.299589 10436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:38.300595 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.302618 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.310787 10437 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:38.290809 12928 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.300256 10436 raft_consensus.cc:606] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:38.314043 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.321026 10436 raft_consensus.cc:614] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:38.317500 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.314862 10434 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:38.322754 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.271410 12917 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:38.339731 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:38.345764 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62539.7us
I20260709 12:10:38.338696 10433 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:38.339736 10438 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:38.340210 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 186945us
I20260709 12:10:38.340750 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.343434 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.347772 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.349339 10434 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:38.346933 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.351678 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.353401 10431 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:38.354079 10432 raft_consensus.cc:2410] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:38.355531 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:38.359565 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.188603 12680 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.360605 10581 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:38.366925 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.137142 10512 rpcz_store.cc:276] 0709 12:10:36.311605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.506330 (+1194725us) service_pool.cc:224] Handling call
0709 12:10:37.506945 (+   615us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:10:37.512770 (+  5825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.512829 (+    59us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:10:37.520844 (+  8015us) log.cc:844] Serialized 78 byte log entry
0709 12:10:37.555891 (+ 35047us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:37.907387 (+351496us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b1800397060. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c926874bc 00007f2c8f9d548b 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c964d73c7 00007f2c92ae99a7 00007f2c92af90bf
0709 12:10:37.907561 (+   174us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.119398 (+211837us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.127457 (+  8059us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.131864 (+  4407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.136364 (+  4500us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.136428 (+    64us) raft_consensus.cc:1685] finished
0709 12:10:38.136442 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.136615 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":222739,"prepare.queue_time_us":764777,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":90,"raft.queue_time_us":21083,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":6728,"spinlock_wait_cycles":6934272,"thread_start_us":76451,"threads_started":2,"wal-append.queue_time_us":88575}
I20260709 12:10:38.373732 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.374483 10433 raft_consensus.cc:2410] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:38.350106 12917 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.377501 10509 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:38.369007 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.361322 12936 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:38.383307 10434 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:10:38.384068 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.387463 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.389019 10511 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.390656 10429 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.394068 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:38.397920 11178 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.760s (SENT)
I20260709 12:10:38.398186 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:38.398851 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 127306us
I20260709 12:10:38.399744 12775 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:38.399886 11178 leader_election.cc:336] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.815s (SENT)
I20260709 12:10:38.400041 12508 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:38.400391 12775 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.405280 12775 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:38.407191 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:38.233688 12514 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:38.418659 12936 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:38.419978 12936 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.421603 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:38.428104 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.432103 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.433398 10436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:50250
W20260709 12:10:38.433717 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44774.1us
I20260709 12:10:38.433979 10436 raft_consensus.cc:606] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:38.434862 12934 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:38.435266 12508 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:38.435451 12934 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:38.436378 12508 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.436695 12934 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.447206 10438 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:38.453594 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.454257 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.456534 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.460210 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:38.462059 13016 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
I20260709 12:10:38.462639 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.463402 10432 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:38.422775 12917 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.458462 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.465451 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.477429 10438 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:38.478015 12904 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.478484 12904 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.475464 10509 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:38.479279 10438 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:38.482767 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:38.484153 10587 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:38.484419 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:38.485687 10582 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.489032 10432 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:38.478885 12904 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.495555 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.496624 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.475555 12534 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:38.501319 10432 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:38.501894 10438 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:38.475653 12830 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.489573 12940 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:38.490496 12299 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:38.498044 13013 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
W20260709 12:10:38.511574 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.517724 12683 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:10:38.516644 10478 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:38.524379 12299 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:38.556236 12299 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.561774 10581 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:38.486322 12945 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:10:38.506008 12534 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.494553 12949 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.592458 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.594022 10432 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:10:38.524569 12733 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.529598 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:38.597733 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.599154 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.594480 10432 pending_rounds.cc:85] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:38.599670 10429 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:38.564947 12935 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:38.601003 10432 pending_rounds.cc:107] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:38.603009 10432 raft_consensus.cc:2889] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:38.603128 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.603940 10429 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:38.575762 12534 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:38.606513 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:38.607631 13010 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.144s	user 0.003s	sys 0.000s
I20260709 12:10:38.619652 12733 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.622356 12733 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.573727 12809 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:10:38.628654 12809 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:38.629123 12809 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:38.563978 10580 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:38.636374 10433 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:38.642009 10582 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:38.643857 10585 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:38.650631 10437 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.651870 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.653226 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.655161 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.655858 10433 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:38.657023 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.657773 10437 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:38.658679 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.659837 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.661194 13021 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:38.660853 12299 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.663374 13021 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.676174 12940 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.636118 12935 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:38.678082 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.647415 12949 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:38.685261 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.599934 12830 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.690538 10437 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:38.685704 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.690855 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.693806 10433 raft_consensus.cc:2410] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:38.695967 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.630569 12809 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.699393 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.700119 10432 raft_consensus.cc:2376] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6. Re-sending same reply.
I20260709 12:10:38.701105 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.702257 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.703477 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.703804 12904 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.704185 10432 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:38.705538 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.706303 12904 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.631372 12933 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.706699 12680 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:38.716653 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:10:38.720583 13014 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:10:38.721230 12946 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:38.721554 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.721853 12946 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:38.722582 10435 raft_consensus.cc:2410] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:10:38.708612 11178 leader_election.cc:336] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.184s (SENT)
I20260709 12:10:38.722962 12946 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.729341 12756 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.729975 12756 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:38.730579 12664 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.001s
I20260709 12:10:38.731120 12664 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.732380 12756 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.733331 12664 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:38.735618 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.706873 10432 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:10:38.739553 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 318028us
I20260709 12:10:38.755421 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.756649 10433 raft_consensus.cc:2376] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6. Re-sending same reply.
I20260709 12:10:38.758435 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.759624 10429 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:10:38.761804 10438 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:38.758332 10435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:50250
I20260709 12:10:38.763994 10435 raft_consensus.cc:606] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:38.764516 10435 raft_consensus.cc:614] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:38.713747 12830 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.676939 12940 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:38.718200 12299 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.766201 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.776315 10429 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:38.760524 12917 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.788856 10433 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:38.778481 12809 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:38.791257 12758 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:38.791937 12758 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:38.793334 10432 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:38.794601 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.714341 12935 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:38.795491 10432 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:10:38.796173 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.796684 10510 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:38.799695 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.801040 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.804909 12933 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.807929 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.808751 10437 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:38.819046 10437 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:38.821724 10437 pending_rounds.cc:85] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:38.822286 10437 pending_rounds.cc:107] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:38.806911 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:38.811800 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:38.822738 10437 raft_consensus.cc:2889] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:38.782763 12942 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:38.786051 12917 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.828552 10586 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:38.828905 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.832053 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:38.832558 10581 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:38.835264 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.836241 10510 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:38.792450 12758 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:38.978595 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.870108 10505 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:38.905555 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:38.841977 10504 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:38.870564 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:39.057106 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.564s (SENT) [suppressed 33 similar messages]
I20260709 12:10:39.057857 10513 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:38.842044 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:39.016782 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.902263 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:39.001765 10584 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:38.900141 10580 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:38.913120 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.910575 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.897920 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:38.983801 12999 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:10:38.911032 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:39.049711 10434 raft_consensus.cc:1217] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:10:38.913069  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:10:38.914134 12996 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:10:39.040309 12830 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:39.063625 10509 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:38.983515 12706 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:39.049284 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47448.7us
I20260709 12:10:38.910298 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:39.077025 10553 leader_election.cc:336] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.706s (SENT)
I20260709 12:10:39.077387 10507 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:39.078727 10585 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:39.079025 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.017107 12733 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:39.086683 10587 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:39.080116 10434 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:10:39.088706 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.088932 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:39.089207 12830 raft_consensus.cc:1068] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:38.911522 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:38.838218 12917 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.120853 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:39.092670 10505 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:38.871582 12940 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:38.913955 12534 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.096982  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:39.079561 11218 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:39.080405 12758 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.104758 13021 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.106545 10513 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:39.107352 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:39.126145 10508 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.084523 12520 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:10:39.111315 11217 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:39.101586 10579 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:10:39.112859 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.121162 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.131165 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.091282 12706 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:39.104461 13023 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.497s	user 0.003s	sys 0.001s
I20260709 12:10:39.136068 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:39.124827 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:39.126693 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:39.137442 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:38.838011 12942 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:39.139206 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.123056 12830 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:39.147917 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.149469 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:39.151201 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:39.152011 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:39.153568 12717 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:39.093218 12733 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:39.155515 10433 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:39.157002 12942 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:39.157325 13026 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.245s	user 0.006s	sys 0.003s
I20260709 12:10:39.156935 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:39.158179 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 81182.1us
I20260709 12:10:39.158687 12697 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:39.159919 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:39.184795 10585 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.169579 10505 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.172312 12809 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.175320 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:39.125234 12904 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:10:39.183303 10509 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:10:39.187594 10513 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:39.142218 12946 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:10:39.173995 13021 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:39.199116 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:39.147401 12949 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 12:10:39.175633 12758 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.210641 12706 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.214301 10582 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:39.217926 10582 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:39.197330 12866 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.223207 10513 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:39.229709 13030 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:39.230470 13030 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:39.236054 10584 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:39.200953 13021 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:39.238173 12706 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.230947 13030 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:39.278719 13025 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.366s	user 0.001s	sys 0.007s
I20260709 12:10:39.243858 12866 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.278242 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:39.293288 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1354) took 1029 ms. Trace:
I20260709 12:10:39.293982 10434 rpcz_store.cc:276] 0709 12:10:38.263687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.439582 (+175895us) service_pool.cc:224] Handling call
0709 12:10:39.049531 (+609949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:39.091695 (+ 42164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:39.292987 (+201292us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400436a50. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:39.293188 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952545152}
I20260709 12:10:39.162540  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:39.317955 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.228921 12758 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:39.414669 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 266808us
I20260709 12:10:39.350870 12942 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.403574 13021 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:39.444504 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 253634us
I20260709 12:10:39.275480 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.428097 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:39.448349 12830 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:39.454280 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.465435 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.466557 10582 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:39.466895 10578 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:39.273018 12706 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.471150 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:39.471855 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.474157 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:39.476400 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:39.478539 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.479300 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.481305 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.449900 12940 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:39.445147 13031 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:39.485169 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59118.7us
I20260709 12:10:39.488298 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:39.487704 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 212366us
I20260709 12:10:39.489537 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:39.490254 10582 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:39.492954 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.444414 12945 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:10:39.451623 12680 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:10:39.479435 13021 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.483606 12858 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.480479 12697 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:39.480093 12534 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.505466 12706 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.507017 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:41062
I20260709 12:10:39.453609 12962 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:10:39.472673 12733 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:39.449658 12942 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.468545 12904 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:10:39.492410 12866 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.514015 10507 raft_consensus.cc:606] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 12:10:39.532898 10582 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:10:39.533401 10507 raft_consensus.cc:614] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:39.541507 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 208) took 1334 ms. Trace:
I20260709 12:10:39.542768 12706 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:39.550901 12706 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:39.551430 12706 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:39.552410 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63716.7us
I20260709 12:10:39.553104 12942 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.553319 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.557086 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.542021 10436 rpcz_store.cc:276] 0709 12:10:38.207454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.434973 (+227519us) service_pool.cc:224] Handling call
0709 12:10:38.442901 (+  7928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:38.760891 (+317990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:38.760939 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:38.765443 (+  4504us) log.cc:844] Serialized 26 byte log entry
0709 12:10:39.125830 (+360387us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:39.457021 (+331191us) log.cc:844] Serialized 14 byte log entry
0709 12:10:39.541123 (+ 84102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:39.541207 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:39.541254 (+    47us) raft_consensus.cc:1685] finished
0709 12:10:39.541266 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:39.541430 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":285774,"prepare.queue_time_us":130030,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":335984,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":24549,"thread_start_us":685987,"threads_started":3,"wal-append.queue_time_us":417049}
W20260709 12:10:39.574116 13039 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:10:39.576990 12758 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:39.579991 12706 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.582749 12706 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.499543 13031 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.585317 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:39.585541 13031 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.586073 11217 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:39.598415 10508 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:39.616642 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 142648us
I20260709 12:10:39.617285 12946 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:39.554576 10332 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 4 to 5, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:39.630641 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 73675.1us
I20260709 12:10:39.630707 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:39.631695 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.632697 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1762) took 1141 ms. Trace:
I20260709 12:10:39.633142 10579 rpcz_store.cc:276] 0709 12:10:38.491168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.491393 (+   225us) service_pool.cc:224] Handling call
0709 12:10:38.504964 (+ 13571us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:38.568657 (+ 63693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:38.568697 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:38.643263 (+ 74566us) log.cc:844] Serialized 574 byte log entry
0709 12:10:39.101168 (+457905us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b180033b440. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c926874bc 00007f2c8f9d548b 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1
0709 12:10:39.101416 (+   248us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:39.606806 (+505390us) log.cc:844] Serialized 14 byte log entry
0709 12:10:39.612728 (+  5922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:39.612865 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:39.612929 (+    64us) raft_consensus.cc:1685] finished
0709 12:10:39.632460 (+ 19531us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:39.632627 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44101,"raft.queue_time_us":40584,"raft.run_cpu_time_us":587,"raft.run_wall_time_us":36613,"spinlock_wait_cycles":6467328,"thread_start_us":136959,"threads_started":1,"wal-append.queue_time_us":148137}
I20260709 12:10:39.635440 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.635192 10585 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:39.518671 12697 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:39.649197 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.625422 13021 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.652031 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.634761 12680 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:39.655473 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 102253us
I20260709 12:10:39.587082 12733 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:10:39.658210 10505 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:39.660096 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.637689 13047 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:39.665086 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:39.652864 12829 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.651849 13050 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:39.670051 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.673270 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.661178 12866 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.668934 12680 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:39.663192 12928 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:39.662950 13031 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:39.653061 12940 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:10:39.677048 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:39.681325 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:39.656330 13021 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:39.671324 13050 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:39.678850 13048 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 12:10:39.689800 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:39.692143 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.691067 13050 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:39.700034 12928 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:39.700498 13021 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:39.707329 13049 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:39.708159 13050 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:39.709061 12917 consensus_meta.cc:287] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.023s
I20260709 12:10:39.709836 11178 leader_election.cc:400] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 3. Current term is 4.
I20260709 12:10:39.746649 11178 leader_election.cc:403] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:10:39.688606 12829 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:39.711427 10436 pending_rounds.cc:85] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:39.750936 10436 pending_rounds.cc:107] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:39.751358 10436 raft_consensus.cc:2889] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:39.751755 10436 pending_rounds.cc:107] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:10:39.752009 10436 raft_consensus.cc:2889] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:39.713753 10434 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:39.683416 12935 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.717903 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.717000 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.753883 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:39.796444 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:39.798269 10438 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:39.799976 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:39.800721 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82849.8us
I20260709 12:10:39.802605 10434 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:39.802543 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:39.804303 10432 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:10:39.798846 10435 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:39.805356 10437 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:39.692035 12904 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:39.750663 12829 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:39.804005 13030 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.530s	user 0.015s	sys 0.003s
I20260709 12:10:39.750528 12940 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:39.700553 12928 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:39.754969 13049 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:39.802004 12706 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:39.755800 12917 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:39.832279 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.814443 10384 auto_rebalancer.cc:893] Removing replace marker failed: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:10:39.836570 10384 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:10:39.691491 13031 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:39.843134 13043 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 3. Current term is 4.
W20260709 12:10:39.845181 13054 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:10:39.846314 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 166811us
I20260709 12:10:39.832832 13051 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.847292 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:39.847728 13051 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:39.848620 13021 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:39.850580 13051 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:39.850952 12917 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:39.851768 10430 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:39.853082 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:39.853281 12758 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:39.854331 12758 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:39.854707 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:39.833082 13055 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.855392 11218 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.835949 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:39.858325 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:39.816867 12904 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:39.824491 12829 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.856817 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:39.756700 12866 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:10:39.820334 12697 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:39.837172 10384 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:10:39.875654 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:39.834916 12928 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:39.855926 12733 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:10:39.884043 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:39.891175 10583 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:39.884397 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.836524 13049 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:39.886756 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 169879us
I20260709 12:10:39.854712 12758 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:39.893477 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:39.870126 13055 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:39.869486 12904 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.874323 12664 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:39.891831 12935 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:10:39.911621 12697 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:39.912169 12680 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:39.914422 12697 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:39.915939 13055 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:39.916178 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:39.924103 12936 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:39.916642 12664 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:39.883860 13044 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:39.922297 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53272.5us
I20260709 12:10:39.834177 13030 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:39.946324 13053 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:10:39.985965 13062 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.988863 10429 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:39.929751 10432 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:39.920195 13051 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:39.944221 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 95286.8us
I20260709 12:10:39.918521 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:39.948901 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55569.3us
I20260709 12:10:39.956748 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.963415 10582 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:39.966967 12917 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:39.968987 10433 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:39.971779 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:39.974329 10430 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:10:39.998319 10430 pending_rounds.cc:85] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:39.998711 10430 pending_rounds.cc:107] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:39.977711 10434 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:39.999261 10430 raft_consensus.cc:2889] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:39.932392 12706 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:39.984195 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:39.938431 13021 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:40.016976 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98574.5us
I20260709 12:10:39.975694 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:40.022059 10435 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:40.006359 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:39.986544 13062 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:39.987365 13030 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.045275 11213 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:39.982888 13060 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:39.987974 13044 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:39.993170 13051 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:40.007926 12706 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:40.055195 12942 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:40.033346 12595 raft_consensus.cc:2764] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:10:40.040119 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:40.023132 12917 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:40.065112 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 74410.7us
I20260709 12:10:40.110525 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:40.065675 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 83535.3us
I20260709 12:10:40.064019 10583 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:40.089018 10586 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:40.073827 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:40.079411 13044 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:40.088165 13059 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.047094 12758 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.118510 12949 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:40.096007 13030 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:40.078367 12706 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:40.159356 12940 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.160017 13066 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:10:40.070173 13050 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:40.212358 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 101975us
I20260709 12:10:40.229048 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.099656 12942 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.219261 11217 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:40.221843 13030 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:40.108151 13061 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:40.161235 13031 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.068882 13021 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.101272 12664 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.210901 11218 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
W20260709 12:10:40.254944 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 167228us
I20260709 12:10:40.241559 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.223024 13059 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.230166 12949 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.151412 13055 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.248111 12830 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:40.253093 13030 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:40.298682 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69718us
W20260709 12:10:40.321841 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 339362us
I20260709 12:10:40.282086 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:40.358970 11178 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.868s (SENT)
I20260709 12:10:40.367261 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:40.369026 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:40.370851 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 88858.8us
I20260709 12:10:40.243942 13051 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.364498 11213 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:40.373076 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:40.240514 12829 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.356045 12733 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:40.362910 12680 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.390493 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.395321 10512 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:40.399168 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.429620 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 56643.8us
I20260709 12:10:40.377640 13062 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:40.481209 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 122274us
I20260709 12:10:40.427704 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:40.415035 12946 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.770s	user 0.005s	sys 0.009s
I20260709 12:10:40.263343 13061 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.379042 13070 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:40.483278 11177 leader_election.cc:336] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.631s (SENT)
W20260709 12:10:40.490530 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 122358us
I20260709 12:10:40.498140 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:40.495910 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68286.5us
W20260709 12:10:40.500875 13072 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:10:40.415613 12917 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:40.508596 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:40.431299 13043 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.512148 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:40.520686 10582 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:40.496102 12946 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:40.417155 12680 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:40.532348 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 117414us
I20260709 12:10:40.549067 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:40.516788 12936 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:10:40.487212 13031 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:40.557380 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58787.6us
I20260709 12:10:40.537544 10332 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 2 to 3, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:40.563565 11220 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:40.515058 13070 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:40.520732 13065 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:10:40.531292 13049 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.627s	user 0.019s	sys 0.002s
I20260709 12:10:40.574733 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.578877 12942 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.579056 12299 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.576743 11212 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:40.579602 12942 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:40.580013 12942 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:40.580369 12942 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:40.582906 13047 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:40.583398 13047 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:40.584411 13047 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.585870 13047 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:40.586211 10435 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.553s	user 0.015s	sys 0.005s
I20260709 12:10:40.587322 10435 raft_consensus.cc:2410] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:40.589236 13083 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:40.589877 13081 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:40.555828 12917 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:40.590350 12904 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 12:10:40.555480 13069 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:40.600239 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 91734.7us
W20260709 12:10:40.599715 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50744.1us
W20260709 12:10:40.611980 10479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.004s	sys 0.010s
W20260709 12:10:40.611927 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47702.3us
I20260709 12:10:40.613026 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:40.614591 10479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.004s	sys 0.010s
W20260709 12:10:40.615319 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 575273us
I20260709 12:10:40.618875 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:40.598150 13083 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:40.616796 12928 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.714s	user 0.018s	sys 0.002s
I20260709 12:10:40.629557 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.571031 13078 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:10:40.631073 12917 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:40.631954 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 90156.2us
I20260709 12:10:40.637465 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.625689 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.635244 13059 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:40.636700 11215 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:40.628918 12659 raft_consensus.cc:2764] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:10:40.650779 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:40.652318 10579 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:10:40.577010 13049 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.655103 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.638157 12917 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:40.612008 12904 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.637076 12829 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.665491 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.648904 12830 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:40.672629 13082 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:40.682137 13089 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.696624 12733 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:40.696408 13044 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.697176 12733 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:40.676359 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.705994 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.678507 12904 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.679191 12680 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.680106 12917 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.681692 13091 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:40.682135 13080 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:10:40.682273 11219 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:40.682945 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:40.667088 13070 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.633879 12928 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.688242 11214 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:10:40.722798 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46593.7us
I20260709 12:10:40.718252 12940 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:40.698839 13044 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.751641 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:40.759675 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53800.8us
W20260709 12:10:40.759819 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.761428 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 151185us
I20260709 12:10:40.762454 10578 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:40.719039 13091 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:40.727248 13089 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:40.747164 13055 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:40.767642 11217 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.511s	user 0.017s	sys 0.005s
I20260709 12:10:40.764724 13091 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:40.784425 13095 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:40.790825 13095 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:40.791965 13061 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.792104 13095 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.794824 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.796319 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:40.798013 12664 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.800133 12534 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:40.797871 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:40.801409 12962 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:40.801821 12962 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:40.802178 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:40.802508 12969 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:40.802909 12969 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:40.803539 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:40.803439 12969 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:40.804709 12928 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:40.805470 12697 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:40.805864 12697 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:40.806232 12697 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:40.807390 13098 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:40.808272 13098 consensus_queue.cc:1243] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:40.811273 13044 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:40.812063 13047 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:40.718573 13083 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.812443 13049 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:40.783423 11217 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:40.809973 13098 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:10:40.815553 13098 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:10:40.815899 12706 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.586s	user 0.013s	sys 0.000s
I20260709 12:10:40.815986 13098 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:40.816413 12706 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.817255 13098 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:40.818434 13075 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.390s	user 0.000s	sys 0.002s
I20260709 12:10:40.822855 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:40.823300 10437 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:40.803251 13097 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:40.826274 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 326750us
I20260709 12:10:40.833734 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.833979 13082 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.835899 13030 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:40.836663 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.837428 10433 raft_consensus.cc:2368] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7.
I20260709 12:10:40.838328 13030 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:40.838912 12940 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:40.839397 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.826382 13097 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:40.840657 13097 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:40.847249 13049 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.847874 13049 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.850129 13049 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:40.855576 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49063.4us
I20260709 12:10:40.855814 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:40.856158 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.857126 11220 raft_consensus.cc:2410] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:10:40.856940 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:40.818923 13098 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:40.818863 12706 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:40.858963 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.862515 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:40.863389 10513 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:40.863533 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.857563 12946 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:40.864262 12946 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:40.864982 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:40.852160 12928 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:10:40.865267 12946 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.865756 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:40.871111 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.872509 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.873898 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:40.875269 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.877449 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.866724 12946 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:40.859032 11219 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:40.865276 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.881184 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.881321 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.868031 12664 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:40.919518 12664 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:40.919968 12664 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:40.922224 12866 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:40.882284 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.860924 13091 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.922835 12866 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.883430 10505 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:10:40.886502 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.925182 11213 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:40.896813 11214 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:40.925594 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.926319 10507 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 2.
I20260709 12:10:40.894198 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:40.927234 10579 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:40.928467 13049 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.913697 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.928926 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.929109 13049 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.930341 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:40.931777 11213 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEARNER]: Advancing to term 6
I20260709 12:10:40.918509 13061 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.933384 13061 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.933702 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:40.934708 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:40.935503 10430 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:40.935573 13061 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:40.936394 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:40.937306 10430 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:40.937631 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:40.938647 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.939569 13073 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.866972 13055 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.802698 12962 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:40.887317 10509 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:40.928414 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:40.931677 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:40.925120 12866 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:40.932641 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:40.925120 13091 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:40.955670 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.993417 11218 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:40.951683 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.994292 10435 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:40.955312 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.952382 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:40.993582 10431 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:40.941987 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:40.871052 12940 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:40.933735 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:40.946751 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.994665 10432 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:40.896085 12942 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.996824 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:40.997174 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55280.8us
I20260709 12:10:40.998737 10430 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:41.003340 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:41.000083 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:40.952765 13029 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:10:41.000339 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.000983 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:41.002316 13104 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:10:41.003557 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:40.999725 13062 raft_consensus.cc:2764] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:10:41.004556 10432 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.006160 12830 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.007091 10585 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 4: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:41.007262 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:41.007817 10431 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:41.011396 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.015110 10430 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:40.931604 13049 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.006248 12733 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.017088 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:41.017596 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:40.941229 12928 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.020898 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:40.947898 13043 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:40.997829 12940 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.028788 10509 raft_consensus.cc:2410] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:41.026101 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.016242 12829 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:10:41.033524 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:41.023679 13055 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:41.006937 12680 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:10:41.048385 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.053092 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.049798 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.035840 13049 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:41.054096 10587 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.054657 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.012033 12664 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:40.944957 13073 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:41.032804 12940 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:41.032305 13059 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.067341 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.068096 10582 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.048247 12962 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:41.061250 10553 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.564s (SENT)
I20260709 12:10:41.062076 13029 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:41.061847 12866 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:10:41.065583 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.083207 10585 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:41.066478 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.084872 10579 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:41.070333 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.060758 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:41.072850 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:41.073382 10587 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:41.100095 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:41.101418 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:41.075706 10432 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:41.056638 12680 consensus_queue.cc:1243] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:41.034096 12928 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.087678 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:41.089295 10509 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:41.119985 10510 raft_consensus.cc:2410] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:10:41.119915 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58724us
I20260709 12:10:41.073884 13029 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.096256 12942 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:41.120725 10553 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.603s (SENT)
I20260709 12:10:41.132711 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:41.098739 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:41.107493 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46828.3us
I20260709 12:10:41.107607 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.108951 12866 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.109779 12969 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.110123 13073 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.134851 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.136338 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.110510 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:41.137367 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.921s (SENT)
I20260709 12:10:41.111904 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:41.113045 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:41.114773 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.116066 11213 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:41.143224 10506 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:41.143181 12858 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:41.143460 13089 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:41.144644 13049 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.145495 13049 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.117043 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.148176 12758 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.067342 11214 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:10:41.121650 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.151091 13055 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.152272 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 1782) took 1186 ms. Trace:
I20260709 12:10:41.152678 10437 rpcz_store.cc:276] 0709 12:10:39.965938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.971449 (+  5511us) service_pool.cc:224] Handling call
0709 12:10:41.152014 (+1180565us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b54003539d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:41.152174 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897389440}
I20260709 12:10:41.153853 12969 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.156384 12928 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:41.160303 12733 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.160055 12697 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.133737 10579 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:41.137544 10511 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:41.110917 13097 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.191946 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:41.138964 10505 raft_consensus.cc:2410] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:10:41.139846 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.141949 12595 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.142095 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:41.194825 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52808.1us
I20260709 12:10:41.167933 12664 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:41.134624 12866 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.135054 12680 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.131582 13029 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.170439 12917 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.175565 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.176339 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.176657 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.176692 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:41.238703 10584 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.244618 11220 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:41.243005 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66391.4us
I20260709 12:10:41.179975 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.199645 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.195276 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:41.246282 10579 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
W20260709 12:10:41.246506 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51389.2us
I20260709 12:10:41.125411 10582 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:41.203156 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 381) took 1146 ms. Trace:
I20260709 12:10:41.201963 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.207085 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.205462 12866 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.248319 10512 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:41.248931 10583 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:41.210811 10510 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:41.212261 13097 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.244521 10513 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:41.195545 13112 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.217387 10509 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:41.223553 13107 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.167s	user 0.002s	sys 0.001s
I20260709 12:10:41.207139 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.224687 12940 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:41.204958 13071 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.006s	sys 0.000s
I20260709 12:10:41.233613 13062 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.242452 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:41.207386 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.247812 11221 rpcz_store.cc:276] 0709 12:10:40.056437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.057526 (+  1089us) service_pool.cc:224] Handling call
0709 12:10:40.596336 (+538810us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:41.191728 (+595392us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bae9c 00007f2c8f9baa5b 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:41.203076 (+ 11348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21943936}
W20260709 12:10:41.255174 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 76564.2us
W20260709 12:10:41.263435 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 83601.2us
I20260709 12:10:41.277631 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:41.278162 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.278481 11221 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:41.279560 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.268059 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:41.281781 13043 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.282222 13043 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.196815 13055 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.281872 10438 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:10:41.300024 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:41.300374 10438 pending_rounds.cc:85] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:41.300663 10438 pending_rounds.cc:107] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:41.300760 10508 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:41.300972 10438 raft_consensus.cc:2889] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:41.196985 13061 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.251436 12946 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:41.295737 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.259840 12664 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.273561 12942 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:41.266353 13112 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.202971 12595 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:41.271203 13071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.006s	sys 0.000s
I20260709 12:10:41.312860 10505 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:41.319978 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.299959 13043 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.302037 12758 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:41.313767 12946 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.319675 12664 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.335465 13102 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:41.323051 13031 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.339617 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.340297 10584 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
W20260709 12:10:41.310559 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 62970.9us
I20260709 12:10:41.308604 13081 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:10:41.325402 12949 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:41.312882 13062 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.326645 13112 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.351111 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.317476 13061 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.353259 10582 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:41.353544 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:41.311753 13055 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.355556 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:41.333851 13030 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.339977 12946 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:41.342741 12664 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.361809 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.332043 13066 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:10:41.368778 13122 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:10:41.380054 12940 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:41.381039 12940 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.382362 13043 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:41.356379 12758 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.386755 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.390147 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.419270 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.363538 13081 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.363763 13102 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.420780 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.423426 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1792) took 1471 ms. Trace:
I20260709 12:10:41.427834 12829 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:41.428783 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 75209.2us
I20260709 12:10:41.433094 10578 rpcz_store.cc:276] 0709 12:10:39.951981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.955096 (+  3115us) service_pool.cc:224] Handling call
0709 12:10:39.955487 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:40.763068 (+807581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.423040 (+659972us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b5400256f54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:41.423326 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855337856}
W20260709 12:10:41.435151 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 60413us
I20260709 12:10:41.389142 13030 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:41.439698 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 157439us
I20260709 12:10:41.438007 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.470330 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:41.467801 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 118732us
I20260709 12:10:41.430073 12697 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:41.446965 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 90870.6us
I20260709 12:10:41.421659 13031 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.432415 12942 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:41.420011 13049 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.434767 12829 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:41.504678 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:41.708782 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:41.710901 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.742064 10580 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:41.455087 13102 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.436106 13081 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.447922 13030 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:41.518658 13117 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:10:41.449903 13050 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:41.713276 13062 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:41.742100 12680 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.534s	user 0.014s	sys 0.014s
I20260709 12:10:41.750041 10578 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:41.748612 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 281258us
I20260709 12:10:41.763779 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.783777 13050 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.784416 13050 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.784855 13050 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.785066 13135 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:41.762099 13030 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.786700 13133 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:41.787150 13133 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:41.788116 13133 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:41.789525 13133 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.782867 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:41.857811 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76643.4us
I20260709 12:10:41.860889 10578 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:41.863443 10578 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:41.865922 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:41.797631 12595 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.768591 13055 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:41.770342 12942 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:41.847162 13049 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:41.891609 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 183067us
W20260709 12:10:41.895262 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 419405us
I20260709 12:10:41.896415 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:41.897833 10578 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:41.851970 12680 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.869313 13065 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:41.898065 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:41.867525 13122 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.848490 13047 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:10:41.946077 10437 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:41.948962 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:41.950084 10432 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:41.951167 10436 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:41.956692 10433 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:41.960032 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:41.961738 10432 pending_rounds.cc:85] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:10:41.962088 10432 pending_rounds.cc:107] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:41.962353 10432 raft_consensus.cc:2889] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:41.964599 13029 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.729s	user 0.010s	sys 0.015s
I20260709 12:10:41.965131 13029 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:41.966320 10580 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:41.966869 12595 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:41.940153 13122 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.967442 12595 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:41.967916 13122 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.967470 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:41.968956 12595 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:41.970592 12659 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:41.971036 10583 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:41.975746 13029 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:41.976670 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:41.980072 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:41.981830 12866 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.717s	user 0.014s	sys 0.004s
I20260709 12:10:41.996402 13050 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.001372 12866 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.001912 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:41.982522 13102 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:41.934640 13055 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:41.030387  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:41.982853 12758 consensus_meta.cc:287] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 12:10:42.002645 10435 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.953680 13125 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:41.900168 13078 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:10:41.953537 13043 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.043673 10436 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:41.956020 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:42.074739 10578 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:42.086129 10435 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:42.087824 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:41.941491 13049 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.044620 12758 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:42.076830 12946 consensus_meta.cc:287] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.003s
I20260709 12:10:42.003716 12659 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.005477 13030 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.075373 13125 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:42.085361 12595 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.166882 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 302) took 1445 ms. Trace:
I20260709 12:10:42.169880 10586 rpcz_store.cc:276] 0709 12:10:40.721410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.721696 (+   286us) service_pool.cc:224] Handling call
0709 12:10:40.750623 (+ 28927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:40.776285 (+ 25662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:40.776320 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:40.776365 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:40.776396 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.165337 (+1388941us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b5400335254. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:42.165419 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:42.166785 (+  1366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58420480}
I20260709 12:10:42.172874 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:42.173667 10435 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:42.173974 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 703733us
W20260709 12:10:42.165633 12917 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.923s	user 0.016s	sys 0.005s
I20260709 12:10:42.176584 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:42.208312 10429 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:42.212870 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:42.214728 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:42.216493 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:42.219199 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:42.221022 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:42.221904 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:42.224710 10582 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.035s	sys 0.037s
W20260709 12:10:42.225100 10582 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.035s	sys 0.037s
W20260709 12:10:42.198184 13112 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.833s	user 0.015s	sys 0.000s
I20260709 12:10:42.252444 13139 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:42.263741 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:42.267060 12659 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:42.163378 12946 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.235045 12917 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:42.228067 12758 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:42.248404 13050 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.280622 12706 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:42.309804 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:42.280952 13139 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:42.281050 12659 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:42.282456 12829 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:42.283126 12946 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.288214 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1787) took 2311 ms. Trace:
I20260709 12:10:42.311262 10438 rpcz_store.cc:276] 0709 12:10:39.976993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.977150 (+   157us) service_pool.cc:224] Handling call
0709 12:10:39.977537 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.312024 (+1334487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.287837 (+975813us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400438354. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:42.288097 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2998515968}
I20260709 12:10:42.312849 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:42.313475 10438 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:42.314465 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:42.290128 13090 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:42.295801 13051 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:42.299290 13141 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:42.279740 10435 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:42.221351 13102 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:42.253325 13112 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.306475 12917 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:42.301084 12866 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:42.336494 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.014s	sys 0.021s
I20260709 12:10:42.319459 10438 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
W20260709 12:10:42.396616 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.014s	sys 0.021s
I20260709 12:10:42.397375 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:42.312278 13139 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:42.316762 13049 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.397979 12917 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:42.311417 12935 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:42.312618 12758 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:42.316777 13050 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:42.318746 12706 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:42.410212 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:42.445211 10438 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:42.318696 12659 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.400123 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:42.448307 10435 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:42.448204 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:42.448908 10436 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:42.434103 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:42.401239 13145 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:42.441000 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:42.403440 12829 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:42.443499 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:42.463495 10578 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:42.447104 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:42.447223 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:42.401005 13139 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:42.403301 13029 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:42.465179 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:42.405540 13055 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:42.439103 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.010s	sys 0.045s
I20260709 12:10:42.459844 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:42.468814 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.010s	sys 0.045s
I20260709 12:10:42.395982 13141 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.466077 10436 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:42.465910 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:42.318408 13125 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.481693 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:42.473999 13055 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.469789 13029 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.489632 13055 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.489915 13029 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:42.490198 12664 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.097s	user 0.016s	sys 0.002s
I20260709 12:10:42.491184 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:42.492049 10583 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:42.467756 13140 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.431540 12935 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.449997 13050 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.497792 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:42.500396 10584 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:42.490733 12664 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.470485 13145 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.521485 12866 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.533371 12866 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.523164 13151 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:42.535578 12935 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.524349 12706 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:42.521579 12758 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.541491 13145 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.535571 13151 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.554339 10583 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
W20260709 12:10:42.555441 13143 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:10:42.556460 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:42.557104 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1800) took 1085 ms. Trace:
I20260709 12:10:42.557495 13055 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:42.559216 13129 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:42.559907 13129 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:42.561143 13129 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.511513 13140 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.558472 13055 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.562930 12659 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:42.577281 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 1.39818e+06us
W20260709 12:10:42.578627 11219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 385) took 2512 ms (client timeout 1669 ms). Trace:
W20260709 12:10:42.579252 11219 rpcz_store.cc:269] 0709 12:10:40.066762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.066920 (+   158us) service_pool.cc:224] Handling call
0709 12:10:40.511916 (+444996us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:42.578278 (+2066362us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54004588d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:42.578545 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390866944}
I20260709 12:10:42.521463 13112 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.557672 10587 rpcz_store.cc:276] 0709 12:10:41.471628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.472214 (+   586us) service_pool.cc:224] Handling call
0709 12:10:41.472635 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:41.725446 (+252811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:41.725479 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:41.744966 (+ 19487us) log.cc:844] Serialized 26 byte log entry
0709 12:10:42.198036 (+453070us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:42.198300 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.198414 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:42.556818 (+358404us) raft_consensus.cc:1685] finished
0709 12:10:42.556852 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:42.557022 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":246906,"raft.queue_time_us":267807,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":10174,"thread_start_us":398621,"threads_started":1,"wal-append.queue_time_us":668914}
W20260709 12:10:42.520387 13122 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.001s
W20260709 12:10:42.581784 10437 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.006s	sys 0.016s
W20260709 12:10:42.591737 10437 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.007s	sys 0.016s
W20260709 12:10:42.582170 10585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.011s
W20260709 12:10:42.592986 10585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.013s	sys 0.011s
I20260709 12:10:42.594749 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1799) took 1123 ms. Trace:
I20260709 12:10:42.595371 10585 rpcz_store.cc:276] 0709 12:10:41.471315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.473367 (+  2052us) service_pool.cc:224] Handling call
0709 12:10:41.719178 (+245811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.720341 (+  1163us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:41.898660 (+178319us) log.cc:844] Serialized 14 byte log entry
0709 12:10:42.593946 (+695286us) log.cc:844] Serialized 14 byte log entry
0709 12:10:42.594268 (+   322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:42.594329 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:42.594403 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.594501 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:42.594658 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":214421,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"thread_start_us":845334,"threads_started":2,"wal-append.queue_time_us":604426}
W20260709 12:10:42.587387 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 272502us
W20260709 12:10:42.600495 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:42.598206 13069 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.959s	user 0.026s	sys 0.026s
I20260709 12:10:42.574466 13141 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.565476 13140 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:42.573060 12866 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.627945 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:42.611408 13145 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:42.643936 10578 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:42.646262 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:10:42.619597 13069 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.987s	user 0.027s	sys 0.026s
I20260709 12:10:42.648130 12534 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:42.649012 13070 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:42.650235 12946 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:42.667694 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:42.654567 12945 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.655990 10511 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:42.611826 13122 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:42.630076 13055 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.598620 12697 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
W20260709 12:10:42.657069 10429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1416) took 1723 ms (client timeout 1930 ms). Trace:
I20260709 12:10:42.674769 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:42.675518 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:42.676244 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:42.656051 13097 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.874370 10513 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:42.909260 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:42.663051 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1797) took 1192 ms. Trace:
I20260709 12:10:42.678773 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:42.912624 10431 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:42.705778 10579 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:42.957104 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 295451us
I20260709 12:10:42.812685 10437 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:42.759692 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:42.709334 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:42.908262 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:42.853010 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:42.818339 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1486) took 1642 ms. Trace:
I20260709 12:10:42.804790 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1485) took 1629 ms. Trace:
W20260709 12:10:42.815867 11220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 335) took 1945 ms (client timeout 1652 ms). Trace:
I20260709 12:10:42.838805 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.005053 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1793) took 1055 ms. Trace:
W20260709 12:10:42.713312 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 243859us
W20260709 12:10:43.006469 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 153595us
W20260709 12:10:42.677212 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 165508us
I20260709 12:10:43.006333 10509 rpcz_store.cc:276] 0709 12:10:41.175648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.190370 (+ 14722us) service_pool.cc:224] Handling call
0709 12:10:41.190833 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.265841 (+ 75008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.818071 (+1552230us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b54001982d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:42.818263 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40996224}
I20260709 12:10:42.958480 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.008015 10431 rpcz_store.cc:276] 0709 12:10:41.949161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.949499 (+   338us) service_pool.cc:224] Handling call
0709 12:10:42.912141 (+962642us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9baf8c 00007f2c8f9baef6 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:42.912363 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.004778 (+ 92415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.004987 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8335872}
I20260709 12:10:42.789472 10435 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:43.007609 10510 rpcz_store.cc:276] 0709 12:10:41.175356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.175476 (+   120us) service_pool.cc:224] Handling call
0709 12:10:41.175828 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.249562 (+ 73734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.804398 (+1554836us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b5400198cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:42.804689 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79954816}
I20260709 12:10:42.839569 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:42.855615 13070 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:42.758677 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.008822 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:43.007774 11220 rpcz_store.cc:269] 0709 12:10:40.870042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.870234 (+   192us) service_pool.cc:224] Handling call
0709 12:10:42.815592 (+1945358us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54004bee54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:42.815758 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666706304}
I20260709 12:10:42.818619 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.017383 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.013388 13070 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.019490 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.022814 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.001722 10582 rpcz_store.cc:276] 0709 12:10:41.470677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.472690 (+  2013us) service_pool.cc:224] Handling call
0709 12:10:41.472987 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.473208 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:42.225470 (+752262us) log.cc:844] Serialized 14 byte log entry
0709 12:10:42.662625 (+437155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:42.662689 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:42.662760 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.662837 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:42.662964 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242772,"prepare.queue_time_us":389856,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":9773,"raft.queue_time_us":1394,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":8416,"thread_start_us":944306,"threads_started":2,"wal-append.queue_time_us":491148}
W20260709 12:10:42.993693 10429 rpcz_store.cc:269] 0709 12:10:40.933370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.933500 (+   130us) service_pool.cc:224] Handling call
0709 12:10:42.656799 (+1723299us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003534d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:42.656994 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2781607680}
I20260709 12:10:43.031535 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1804) took 1049 ms. Trace:
I20260709 12:10:43.032002 10437 rpcz_store.cc:276] 0709 12:10:41.981777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.811995 (+830218us) service_pool.cc:224] Handling call
0709 12:10:42.812438 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.025729 (+213291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.031457 (+  5728us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.033455 10437 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:42.667346 13098 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.052826 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.006196 10511 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:42.911612 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:42.842067 13115 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.510s	user 0.004s	sys 0.013s
W20260709 12:10:42.977787 13149 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:10:43.052305 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43590.8us
W20260709 12:10:42.839414 12595 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.005s
I20260709 12:10:42.669399 13069 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:42.799412 13155 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.066785 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:42.791101 13061 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:42.697966 12946 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:42.790906 12664 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:43.103379 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.015012 13097 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.029470 12935 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:10:43.103960 13070 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.660123 13145 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.677618 13151 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.820897 13030 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.104702 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.106050 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:42.679797 12945 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.057871 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:43.104566 13125 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.128207 10509 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:42.743760 13059 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.128743 12697 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:42.860389 13071 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:43.128763 13150 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.191560 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.192134 10509 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:43.191192 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:43.130856 13051 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.002s
I20260709 12:10:42.824361 13122 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:42.717492 13050 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:42.814298 13081 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.711951 12706 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.185212 13125 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:43.201156 13125 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.200995 12706 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:42.857527 13044 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.973107 13169 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.931756 13066 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.813114 13136 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:42.857806 13112 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:43.021433 13129 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.222677 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1807) took 1240 ms. Trace:
I20260709 12:10:43.223235 10431 rpcz_store.cc:276] 0709 12:10:41.982734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.022584 (+1039850us) service_pool.cc:224] Handling call
0709 12:10:43.127960 (+105376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.217594 (+ 89634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.217653 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.217731 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:43.222301 (+  4570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.222406 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.222602 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75347,"raft.queue_time_us":79815}
I20260709 12:10:42.966650 12928 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.226706 13136 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:42.716933 13135 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.228056 13135 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.230376 13135 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:42.994913 13140 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.230512 13136 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.029940 12829 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:42.746456 13082 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:43.155066 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.156852 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:43.248591 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 91832.2us
W20260709 12:10:43.160216 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 490366us
I20260709 12:10:43.246857 12829 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:43.240589 13066 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.250092 12829 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.250787 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:43.252367 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:43.252535 12945 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:43.160972 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1813) took 1152 ms. Trace:
W20260709 12:10:43.160256 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 138382us
I20260709 12:10:43.253660 13066 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:43.182282 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 124520us
I20260709 12:10:43.254189 10478 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:42.911317 13139 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.187747 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.254750 13177 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:43.255395 11220 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:43.255642 13177 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:42.720913 12917 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.256243 13177 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.196192 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:43.257230 10438 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:43.257393 11219 raft_consensus.cc:2376] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6. Re-sending same reply.
I20260709 12:10:43.196616 13059 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.257969 12917 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.195286 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1794) took 1245 ms. Trace:
I20260709 12:10:43.258461 13168 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:43.204561 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 293056us
I20260709 12:10:43.259160 13168 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.207696 13044 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.226512 12758 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:43.227155 12928 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.223001 13049 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.231120 13140 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.146886 10429 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:43.152491 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1429) took 2147 ms. Trace:
I20260709 12:10:43.222576 13169 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.258071 10437 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:43.220578 12942 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.233480 12946 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.193652 12664 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.234074 13129 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.165104 13155 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.289355 12829 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.291111 13155 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.191165 13156 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.291859 12829 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.191609 12866 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:43.236447 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.292208 13156 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.236593 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.281407 12664 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.295542 12664 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.203152 13150 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.237121 10332 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:43.238240 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.238973 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:43.304188 11213 raft_consensus.cc:2376] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7. Re-sending same reply.
I20260709 12:10:43.305261 13150 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.274755 12659 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.306502 12659 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.308372 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1801) took 1327 ms. Trace:
I20260709 12:10:43.309072 12659 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:43.310081 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1813) took 1062 ms. Trace:
I20260709 12:10:43.203800 13071 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.218575 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:43.247700 13082 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.256742 13139 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.190165 10513 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:43.222448 13112 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:43.265195 10429 pending_rounds.cc:85] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 1
I20260709 12:10:43.332749 13071 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.332963 10429 pending_rounds.cc:107] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:10:43.333338 10429 raft_consensus.cc:2889] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:43.334185 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1810) took 1345 ms. Trace:
I20260709 12:10:43.334205 13112 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.333976 13082 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.334610 10429 rpcz_store.cc:276] 0709 12:10:41.989235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.028846 (+1039611us) service_pool.cc:224] Handling call
0709 12:10:43.146306 (+117460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.333854 (+187548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.334115 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.335304 13082 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:43.336601 10429 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:10:43.337325 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1433) took 1086 ms. Trace:
I20260709 12:10:43.337654 10429 rpcz_store.cc:276] 0709 12:10:42.250829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.335897 (+1085068us) service_pool.cc:224] Handling call
0709 12:10:43.336341 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.337028 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.337267 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:43.266629 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.340425 13140 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.253149 10579 rpcz_store.cc:276] 0709 12:10:42.008854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.024356 (+ 15502us) service_pool.cc:224] Handling call
0709 12:10:42.024904 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.004849 (+979945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.160711 (+155862us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b5400334854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:43.160905 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565312384}
I20260709 12:10:43.340996 13140 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.341535 13140 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.341871 12299 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:43.342806 12299 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:43.343729 12299 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:43.344512 12928 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.345199 13122 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:43.233003 13097 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.274197 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.233316 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.279927 10512 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:43.346468 10509 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:43.280321 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.270414 10510 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:43.280296 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:43.258865 10432 rpcz_store.cc:276] 0709 12:10:41.949880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.951300 (+  1420us) service_pool.cc:224] Handling call
0709 12:10:41.961478 (+ 10178us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:41.963694 (+  2216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:41.963742 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:41.964348 (+   606us) log.cc:844] Serialized 52 byte log entry
0709 12:10:42.326272 (+361924us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:42.810354 (+484082us) log.cc:844] Serialized 14 byte log entry
0709 12:10:42.873185 (+ 62831us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.185254 (+312069us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b1800312b80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c92ad7060 00007f2c92ae9716 00007f2c92afa287 00007f2c92afa208
0709 12:10:43.185461 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.185531 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:43.195011 (+  9480us) raft_consensus.cc:1685] finished
0709 12:10:43.195034 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.195220 (+   186us) inbound_call.cc:177] Queueing success response
0709 12:10:43.237678 (+ 42458us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b1800312b80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
Metrics: {"mutex_wait_us":57251,"prepare.queue_time_us":251128,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":92,"raft.queue_time_us":528071,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":85538,"spinlock_wait_cycles":76021888,"thread_start_us":590688,"threads_started":2,"wal-append.queue_time_us":565374}
I20260709 12:10:43.282568 10431 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:10:43.348305 10431 pending_rounds.cc:85] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
W20260709 12:10:43.345899 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 63878.4us
I20260709 12:10:43.348727 10431 pending_rounds.cc:107] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:43.349176 10431 raft_consensus.cc:2889] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:43.283599 12595 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:43.262826 13070 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:10:43.284984 12945 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:43.285804 12936 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:43.267153 10430 rpcz_store.cc:276] 0709 12:10:41.004625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.023134 (+ 18509us) service_pool.cc:224] Handling call
0709 12:10:41.023484 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:41.026562 (+  3078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:41.026594 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:41.033575 (+  6981us) log.cc:844] Serialized 26 byte log entry
0709 12:10:41.206329 (+172754us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:41.206380 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.206451 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:42.841142 (+1634691us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:43.132199 (+291057us) raft_consensus.cc:1685] finished
0709 12:10:43.132234 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.152365 (+ 20131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1187,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3816,"spinlock_wait_cycles":5379840,"thread_start_us":146949,"threads_started":1,"wal-append.queue_time_us":219745}
I20260709 12:10:43.361050 10430 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:10:43.361981 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1438) took 1106 ms. Trace:
I20260709 12:10:43.287070 13059 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.287256 13044 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.288681 13129 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.291800 13177 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.292689 12866 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.305675 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.241482 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.236860 13069 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.239957 13145 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:43.310540 10437 rpcz_store.cc:276] 0709 12:10:42.249408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.155998 (+906590us) service_pool.cc:224] Handling call
0709 12:10:43.156411 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.265752 (+109341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.309779 (+ 44027us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003962d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:43.310013 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233211264}
I20260709 12:10:43.308818 10438 rpcz_store.cc:276] 0709 12:10:41.980881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.481565 (+500684us) service_pool.cc:224] Handling call
0709 12:10:42.481969 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.257651 (+775682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.308081 (+ 50430us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400087ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:43.308296 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782487808}
I20260709 12:10:43.379206 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1814) took 1071 ms. Trace:
I20260709 12:10:43.378849 10437 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:43.379621 10438 rpcz_store.cc:276] 0709 12:10:42.308307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.378149 (+1069842us) service_pool.cc:224] Handling call
0709 12:10:43.378572 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.378800 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.378829 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.378881 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:43.378920 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.379005 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.379137 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30720}
I20260709 12:10:43.249300 13173 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:43.326156 11215 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.328625 12697 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:43.387388 13145 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.388136 12697 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:43.388025 13145 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.388584 12697 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.389601 12697 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.390547 13145 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.330116 13098 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:43.391189 13112 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:10:43.392170 13097 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:43.392359 13167 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:43.392678 13097 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.392829 13167 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.331087 13150 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.393803 13167 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.294873 13156 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.345296 12928 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:43.395434 12928 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.396433 12928 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.391152 12697 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.397589 12928 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:43.347792 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67580.1us
I20260709 12:10:43.283526 12942 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.399600 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.400146 11215 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:43.401513 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.253291 12706 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.403302 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1806) took 1420 ms. Trace:
I20260709 12:10:43.346482 11219 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:43.403743 10436 rpcz_store.cc:276] 0709 12:10:41.982432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.873591 (+891159us) service_pool.cc:224] Handling call
0709 12:10:42.874018 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.203198 (+329180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.203232 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.203276 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:43.203312 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.403007 (+199695us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54002652d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:43.403062 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.403229 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756551936}
I20260709 12:10:43.404965 13066 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.405691 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1444) took 1148 ms. Trace:
I20260709 12:10:43.405561 13066 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.406138 10435 rpcz_store.cc:276] 0709 12:10:42.257131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.469392 (+212261us) service_pool.cc:224] Handling call
0709 12:10:43.405486 (+936094us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54000870d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:43.405618 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705669376}
I20260709 12:10:43.340996 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.352010 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.409029 10579 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:43.409329 13066 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.353345 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.409914 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1803) took 1428 ms. Trace:
W20260709 12:10:43.407953 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67065us
I20260709 12:10:43.411417 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.412209 10579 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:43.412627 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.413223 10511 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:43.326578 12659 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.413560 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:43.414621 13129 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.373610 12945 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.415133 13129 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:43.415657 13129 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.416132 12945 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.419181 12917 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:43.420094 12917 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:43.421447 13169 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:43.421715 12917 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.356360 10429 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:43.356851 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1432) took 1106 ms. Trace:
I20260709 12:10:43.426409 10431 rpcz_store.cc:276] 0709 12:10:42.250525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.280776 (+1030251us) service_pool.cc:224] Handling call
0709 12:10:43.282187 (+  1411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.356546 (+ 74359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.356784 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.410396 10580 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 4: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:43.362426 10430 rpcz_store.cc:276] 0709 12:10:42.255345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.360320 (+1104975us) service_pool.cc:224] Handling call
0709 12:10:43.360668 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.361610 (+   942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.361901 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.345825 13122 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.393774 13097 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.428669 13122 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.410382 10433 rpcz_store.cc:276] 0709 12:10:41.981467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.592903 (+611436us) service_pool.cc:224] Handling call
0709 12:10:43.408431 (+815528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.408655 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.408691 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.408750 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:43.408785 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.408847 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.409716 (+   869us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54000870d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:43.409847 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874673024}
I20260709 12:10:43.430060 13122 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.261883 12758 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.431545 12866 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:43.344612 12299 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:43.355778 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75612.1us
I20260709 12:10:43.355057 13155 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.433063 12758 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.433552 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:43.371836 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 66243.5us
I20260709 12:10:43.375324 10432 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:43.341578 12829 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.435887 12829 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.373154 10505 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:43.438380 12829 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.357312 13061 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.321722 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.439531 10581 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:43.382473 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1815) took 1073 ms. Trace:
I20260709 12:10:43.384567 13173 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.394117 13150 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.440398 10438 rpcz_store.cc:276] 0709 12:10:42.309035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.381008 (+1071973us) service_pool.cc:224] Handling call
0709 12:10:43.381877 (+   869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.382106 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.382138 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.382184 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:43.382220 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.382279 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.382406 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.441134 13173 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.414228 12659 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.442787 13173 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:43.444542 12659 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.432255 12299 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.260603 13168 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.446775 11217 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:43.446241 12299 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.446084 10332 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 4 to 5, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:43.447844 12299 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.448108 10332 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 4 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:43.449992 10332 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:43.402729 12942 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.442334 13150 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.451664 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:43.466776 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.467620 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.469317 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.452072 13140 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.451701 13167 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:43.454840 13049 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:43.380581 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1439) took 1125 ms. Trace:
I20260709 12:10:43.476289 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1434) took 1222 ms. Trace:
I20260709 12:10:43.372381 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:43.470721 13177 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:43.471045 13055 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:43.472071 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1437) took 1217 ms. Trace:
I20260709 12:10:43.472621 13155 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.473143 13145 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:43.489383 13155 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:43.489526 13055 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:43.489904 13155 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.473987 13150 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:43.490125 12942 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:43.491196 12935 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.498319 13145 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.473724 13125 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.474553 13061 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.498397 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.475708 10437 rpcz_store.cc:276] 0709 12:10:42.255667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.378104 (+1122437us) service_pool.cc:224] Handling call
0709 12:10:43.378590 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.380295 (+  1705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.380522 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29440}
I20260709 12:10:43.381465 13139 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:43.464653 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52233.8us
I20260709 12:10:43.479451 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:43.481860 10581 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:43.481474 10432 rpcz_store.cc:276] 0709 12:10:42.255040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.360300 (+1105260us) service_pool.cc:224] Handling call
0709 12:10:43.375011 (+ 14711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.435420 (+ 60409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.471711 (+ 36291us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b54002e4620. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae2a02 00007f2c92adedd6 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:10:43.471991 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8236416}
W20260709 12:10:43.483577 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 113233us
I20260709 12:10:43.475323 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.489934 13055 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.502657 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:43.476831 10429 rpcz_store.cc:276] 0709 12:10:42.253997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.338469 (+1084472us) service_pool.cc:224] Handling call
0709 12:10:43.338823 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.475936 (+137113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.476204 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.500180 10332 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 3, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:10:43.519053 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 202707us
W20260709 12:10:43.525117 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58469.4us
I20260709 12:10:43.542168 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:43.543251 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:43.545063 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:43.546238 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:43.544961 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.550640 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.548619 13184 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:10:43.552380 13180 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:43.553473 13180 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:43.554612 13180 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:43.554877 12928 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:43.555490 12928 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.556154 12928 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:43.556730 13070 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:43.558969 12866 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:43.563441 13176 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:43.522651 13133 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.566485 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.566967 13133 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.567461 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.498322 12942 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.568444 12942 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.569468 13133 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.569730 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.571223 10585 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:43.571774 13125 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:43.576735 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.571700 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.577039 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.566823 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.568307 10504 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.555629 12595 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:43.579401 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.581462 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.581878 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.583109 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.512442 13061 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:43.523061 12758 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:43.586130 10507 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:43.587389 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.590641 12299 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.580415 12829 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.591408 12829 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:43.591313 12299 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:43.591928 12829 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.593820 12299 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.594352 12299 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.580615 12595 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.575107 13071 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:10:43.596015 12595 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.597218 13071 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:43.580732 13150 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:10:43.597589 12595 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.539531 13043 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:43.570780 12942 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:43.599040 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49230.3us
I20260709 12:10:43.585527 12936 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:10:43.545379 13139 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:43.535765 12935 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 12:10:43.605073 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.620348 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.620697 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.621374 10579 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:43.600987 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.622696 10582 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:43.580905 13145 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:43.603986 12866 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:43.579625 13140 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:43.603957 13156 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.587934 13061 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.601832 13043 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.608721 13082 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.637908 13139 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.638180 13082 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.602535 13069 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.639335 13069 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.640554 13082 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.613533 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.642091 10513 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:43.594789 12299 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.578683 13180 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.614343 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.588035 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:43.644801 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.645488 10438 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:43.646488 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.566485 12664 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.647322 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.647965 10431 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:43.650532 10430 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:43.626425 10437 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:43.626363 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.539824 13177 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:10:43.598352 13049 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.658124 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.633173 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.589721 12758 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.634871 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.677358 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.605561 13125 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.682001 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.624022 12659 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.637153 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.597646 13071 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:43.643323 10583 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:43.653698 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.655951 10430 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:43.689282 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.689771 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.689906 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:43.656661 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.691844 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.692454 10431 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:43.684679 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.693485 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.590377 13184 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:43.658805 10508 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:43.658805 10507 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.660036 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.666069 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:43.701591 11215 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:43.674816 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.660558 13069 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:43.676482 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.710259 10512 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:43.676875 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.711066 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:43.711540 10435 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:43.678057 10433 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:43.711704 10506 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:43.677955 10582 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
W20260709 12:10:43.690904 10479 connection.cc:256] RPC call timeout handler was delayed by -1.2698293309999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 12:10:43.712831 10479 connection.cc:256] RPC call timeout handler was delayed by -1.1861293310000747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 12:10:43.713512 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.679049 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.679674 12299 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.603592 12946 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:10:43.631449 12706 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.684343 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:43.720422 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78951.3us
I20260709 12:10:43.642391 13043 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.687203 13150 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:43.694684 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.691035 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.702409 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.700969 10509 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:43.616097 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:43.707113 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:43.726212 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.682031 12758 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.658913 13140 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.707984 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.736712 10432 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:43.660187 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.714258 12299 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.624308 13055 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.694010 10436 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:43.720891 11221 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:43.736539 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:43.749037 10433 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.654803 13156 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.737424 12299 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.715044 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:43.760389 10438 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.700143 13176 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.703099 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:43.743788 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.740410 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.762286 10510 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:43.780488 10435 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:43.793046 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.792845 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:43.794920 12299 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.806284 10585 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:43.808713 12664 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:43.793524 13176 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.809849 13097 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.699687 12659 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.738195 13174 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:43.814802 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.840973 10436 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:43.695873 13125 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:43.724730 13150 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:43.831627 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.855326 12697 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.856314 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:43.857191 10509 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:43.846493 10437 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
W20260709 12:10:43.858582 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47535.7us
I20260709 12:10:43.859555 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.860554 10585 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:43.855340 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.759073 12534 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:10:43.860220 10437 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:43.879962 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.880869 10437 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:43.881733 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.882385 10437 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:43.883167 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.828403 13055 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.743716 13129 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:43.884284 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.891903 10437 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:43.893196 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.893828 10437 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:43.894666 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.888090 10438 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:43.889779 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.896858 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.897658 10438 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:43.905289 10433 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:43.906394 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.908941 10437 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:43.840160 13174 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:43.925930 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.932638 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.933808 10585 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:43.935763 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.936869 10436 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:43.937170 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.938686 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.939431 10582 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.793429 13071 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.721051 13180 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:43.961014 10582 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:43.952742 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:43.967773 12299 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:43.967949 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.951761 10585 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:43.963812 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:43.968706 10582 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:43.969424 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:43.965237 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:43.965531 11221 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:43.973114 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:43.938935 12534 consensus_queue.cc:1243] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:43.982919 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.968168 12299 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:43.983834 12299 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:43.968144 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.977909 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.963636 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:43.995716 10504 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:43.947180 13082 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.997473 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:43.996202 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.017331 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.017668 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:44.018929 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.021379 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.023299 10436 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:44.023567 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:44.025110 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:44.022208 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 148212us
I20260709 12:10:44.027937 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:44.032234 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.033011 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:44.033835 10437 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:44.048794 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.059204 10584 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:44.060845 10584 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:44.053154 13069 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.048986 13184 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:44.066951 13069 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.067803 13184 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.057732 10585 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:44.071234 13069 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.050204 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:44.072876 13069 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:44.074029 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:44.075806 12299 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.095409 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:44.096518 10437 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.099820 13156 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.101014 13198 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.966636 13150 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.101754 12664 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:44.102236 13189 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:44.102555 12664 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:44.057679 13073 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:44.106376 13073 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:44.106791 13073 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.944226 13140 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.035887 13082 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.003635 13177 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
I20260709 12:10:44.036180 13139 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:10:44.022190 13174 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.047255 13194 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:44.120237 12299 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.112471 12659 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.107895 13198 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:44.128362 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54435.2us
W20260709 12:10:44.130455 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 199286us
W20260709 12:10:44.165503 10513 consensus_meta.cc:287] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.001s
W20260709 12:10:44.112058 13195 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:10:44.171649 12595 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.004s
W20260709 12:10:44.124771 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 151729us
I20260709 12:10:44.123394 13194 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:44.122668 12829 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.000s
I20260709 12:10:44.107712 13205 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:44.135985 13139 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:44.117903 13082 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:44.142431 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:44.139611 13122 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.709s	user 0.014s	sys 0.000s
I20260709 12:10:44.171360 13129 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.173625 11178 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:44.246268 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72741.3us
I20260709 12:10:44.177326 10437 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:44.135537 10507 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:44.167363 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:44.249760 10432 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:10:44.166914 10513 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4.
I20260709 12:10:44.150813 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.250835 10508 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:10:44.140628 13187 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:10:44.168366 10585 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:44.148453 10332 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:44.194510 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.253011 10584 raft_consensus.cc:2410] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:10:44.202594 11177 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.674s (SENT) [suppressed 12 similar messages]
W20260709 12:10:44.254179 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51648.2us
W20260709 12:10:44.254868 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:44.204948 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 186430us
I20260709 12:10:44.255939 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.256987 13082 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:44.257694 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.258555 13082 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:44.249656 10504 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:44.124630 13069 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:44.249562 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82293.6us
I20260709 12:10:44.133301 13066 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:44.121227 13174 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.155646 13125 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:44.139477 13155 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.003s
I20260709 12:10:44.266083 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:44.266817 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.299553 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:44.284052 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.145799 13198 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:44.146799 13176 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.168475 13156 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 12:10:44.287788 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:44.244920 12706 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:44.260758 13070 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:44.185266 12928 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.000s
I20260709 12:10:44.106976 12758 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.324201 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.306504 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.308760 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.287317 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.288708 10437 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:44.283658 13049 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:44.186205 13133 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.121s
I20260709 12:10:44.295043 13066 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:44.310587 13122 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:44.293377 13207 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:44.314044 13125 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:44.236824 12829 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:44.367645 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61262.8us
W20260709 12:10:44.372583 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 347563us
I20260709 12:10:44.326558 10512 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:44.381031 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:44.321432 13169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.002s	sys 0.008s
I20260709 12:10:44.387903 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.400400 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.402604 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:44.403100 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:44.404356 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:44.408097 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.401137 10512 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:44.416657 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:44.331868 13049 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.310326 13155 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.370489 13082 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:44.421988 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.374198 13066 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:44.394130 13169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.002s	sys 0.008s
I20260709 12:10:44.424804 10586 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:44.427554 12299 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:44.454159 12299 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:44.454698 12299 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.428844 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:44.425400 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 397529us
I20260709 12:10:44.456710 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.458479 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:44.374861 13207 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:44.433439 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 170317us
I20260709 12:10:44.378713 13125 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.323612 12706 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.440560 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.348450 12928 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.323793 13070 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.417927 12659 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:44.419708 13122 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:44.418202 13177 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:44.417541 13184 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:44.430685 10581 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:44.456833 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:44.384979 13073 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.438082 13049 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:44.460141 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 172438us
I20260709 12:10:44.471937 10430 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:44.423409 13140 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:44.473366 10512 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:44.472865 10505 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:44.474525 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:44.471060 13043 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.710s	user 0.013s	sys 0.000s
I20260709 12:10:44.455118 13156 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:44.488085 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:44.525909 13210 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:10:44.526903 12829 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.530886 12299 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:44.522630 13082 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:44.378126 13194 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:44.535933 10434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1417) took 3584 ms (client timeout 2263 ms). Trace:
W20260709 12:10:44.652140 10434 rpcz_store.cc:269] 0709 12:10:40.934036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.940997 (+  6961us) service_pool.cc:224] Handling call
0709 12:10:44.518524 (+3577527us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b5400264dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:44.518745 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5946297088}
W20260709 12:10:44.652251 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.008s
I20260709 12:10:44.521461 12659 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.472177 13125 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:44.583776 10582 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:44.465090 13207 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.460268 13214 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:44.615437 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 158684us
I20260709 12:10:44.579545 13122 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.578922 13139 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:44.586129 13177 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.603663 13184 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.614298 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:44.644752 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.713670 13184 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:44.715094 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.715826 10505 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:44.644451 13043 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:44.653649 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.006s	sys 0.008s
I20260709 12:10:44.654608 12299 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.700454 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1817) took 1036 ms. Trace:
I20260709 12:10:44.676064 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.723232 10431 rpcz_store.cc:276] 0709 12:10:43.663701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.915161 (+251460us) service_pool.cc:224] Handling call
0709 12:10:43.915558 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.915999 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.916034 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.945563 (+ 29529us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.084209 (+138646us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:44.084269 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.084380 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.700156 (+615776us) raft_consensus.cc:1685] finished
0709 12:10:44.700191 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.700370 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":7824,"thread_start_us":61443,"threads_started":1,"wal-append.queue_time_us":217976}
I20260709 12:10:44.724289 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.645251 13140 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:44.684854 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.645470 13198 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:44.690450 13217 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:10:44.690354 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:44.691102 13139 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.697062 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.700453 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 222) took 1076 ms. Trace:
I20260709 12:10:44.701442 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1514) took 1136 ms. Trace:
I20260709 12:10:44.746918 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:44.749766 10434 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:44.749410 10510 rpcz_store.cc:276] 0709 12:10:43.565267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.568739 (+  3472us) service_pool.cc:224] Handling call
0709 12:10:43.569129 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.780222 (+211093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.701125 (+920903us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540036ef54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:44.701361 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443362688}
I20260709 12:10:44.688675 13122 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:44.707284 13222 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:10:44.708194 13173 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:44.708853 13193 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:44.719841 12758 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:44.717005 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 102809us
I20260709 12:10:44.659266 12659 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:44.770351 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.737176 13145 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:44.737794 13198 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:44.651997 13082 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.736117 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.775899 10505 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.743830 13212 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:44.743220 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 268805us
I20260709 12:10:44.748646 10429 rpcz_store.cc:276] 0709 12:10:43.624421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.625297 (+   876us) service_pool.cc:224] Handling call
0709 12:10:43.625579 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.697943 (+1072364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.697991 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.698053 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:44.698100 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.700098 (+  1998us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400349754. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:44.700170 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.700362 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2465524480}
I20260709 12:10:44.705233 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1827) took 1031 ms. Trace:
I20260709 12:10:44.720145 13177 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.744791 13218 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:44.794198 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:44.709221 13073 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.722481 13156 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.759573 13229 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:44.797549 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:44.737953 13140 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.798627 13140 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.781317 10432 rpcz_store.cc:276] 0709 12:10:43.673845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.152142 (+478297us) service_pool.cc:224] Handling call
0709 12:10:44.249472 (+ 97330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.250204 (+   732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.705014 (+454810us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b5400438350. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:44.705162 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222880000}
I20260709 12:10:44.785971 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.785971 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.729908 10431 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:44.800153 10580 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:10:44.799186 11216 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:44.759652 12706 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.788431 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.820147 11219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 399) took 1320 ms. Trace:
I20260709 12:10:44.768810 13206 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 12:10:44.780385 13224 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.358s	user 0.002s	sys 0.000s
I20260709 12:10:44.764933 11217 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:44.815667 13229 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.830578 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:44.831693 10505 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:44.832163 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:44.832315 13229 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.832774 10506 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.833451 13228 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:44.833916 13228 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:44.834266 13228 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:44.764544 13173 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:44.835548 13066 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.836817 11213 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:44.836491 13173 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.763338 10434 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:44.838389 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:44.793756 13218 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:44.839294 13218 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.789361 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1829) took 1112 ms. Trace:
I20260709 12:10:44.843164 10586 rpcz_store.cc:276] 0709 12:10:43.676557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.677874 (+  1317us) service_pool.cc:224] Handling call
0709 12:10:44.789120 (+1111246us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54001caf54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:44.789285 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609064704}
I20260709 12:10:44.843618 13228 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.844681 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:44.773722 13145 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:44.845830 13145 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:44.845971 13228 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.780061 13082 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:44.805807 13216 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:10:44.790961 13212 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.849997 13212 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.808185 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:44.775471 13198 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:44.792533 13069 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:10:44.851437 10512 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:44.850795 10585 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:44.813637 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 209) took 1200 ms. Trace:
I20260709 12:10:44.794531 13237 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:10:44.817180 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:44.789441 13155 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.787979 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1823) took 1118 ms. Trace:
I20260709 12:10:44.880016 10433 rpcz_store.cc:276] 0709 12:10:43.670001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.124545 (+454544us) service_pool.cc:224] Handling call
0709 12:10:44.124862 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.125111 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:44.375738 (+250627us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.787449 (+411711us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.787532 (+    83us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:44.787603 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.787706 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.787891 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5195,"prepare.queue_time_us":405761,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":7703,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4305,"spinlock_wait_cycles":7522560,"thread_start_us":656164,"threads_started":2,"wal-append.queue_time_us":420272}
I20260709 12:10:44.825369 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:44.820889 12928 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.827706 13049 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.852541 10511 rpcz_store.cc:276] 0709 12:10:43.612786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.612912 (+   126us) service_pool.cc:224] Handling call
0709 12:10:43.613226 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.022423 (+409197us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:44.329130 (+306707us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.654806 (+325676us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.677598 (+ 22792us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.687476 (+  9878us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:44.687538 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.687647 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.813356 (+125709us) raft_consensus.cc:1685] finished
0709 12:10:44.813388 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.813557 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119655,"prepare.queue_time_us":63627,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":162137,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":20369,"thread_start_us":733770,"threads_started":3,"wal-append.queue_time_us":352345}
I20260709 12:10:44.896139 13049 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:44.828439 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:44.897370 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69042.9us
I20260709 12:10:44.828820 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.899387 10510 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
W20260709 12:10:44.829572 13174 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.526s	user 0.004s	sys 0.007s
I20260709 12:10:44.774816 13193 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.832641 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:44.838451 13066 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:44.854444 10432 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.820639 11219 rpcz_store.cc:276] 0709 12:10:43.500142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.500432 (+   290us) service_pool.cc:224] Handling call
0709 12:10:43.542101 (+ 41669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.595077 (+ 52976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.595115 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.595718 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:10:43.838811 (+243093us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b180047fbc0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:44.316587 (+477776us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:44.316648 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.316753 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.800163 (+483410us) raft_consensus.cc:1685] finished
0709 12:10:44.800186 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.820054 (+ 19868us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11897,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4983,"spinlock_wait_cycles":8120192,"thread_start_us":477496,"threads_started":1,"wal-append.queue_time_us":938198}
I20260709 12:10:44.855329 10434 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.813896 13140 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:44.904556 10436 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.009s	sys 0.041s
I20260709 12:10:44.901130 13214 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:44.948630 10436 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.009s	sys 0.041s
I20260709 12:10:44.932497 10505 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:10:44.923910 10333 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 231) took 20071 ms (client timeout 15000 ms). Trace:
I20260709 12:10:44.938827 10506 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:44.929131 13207 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:44.964447 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 215128us
I20260709 12:10:44.878342 13213 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.815218 13156 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:44.965799 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:44.913584 13235 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:45.039979 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1818) took 1375 ms. Trace:
I20260709 12:10:45.052628 10432 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:45.051683 10436 rpcz_store.cc:276] 0709 12:10:43.664459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.057967 (+393508us) service_pool.cc:224] Handling call
0709 12:10:44.103851 (+ 45884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.104249 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.104284 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.104845 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.317023 (+212178us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.949312 (+632289us) log.cc:844] Serialized 14 byte log entry
0709 12:10:45.039594 (+ 90282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.039682 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:45.039732 (+    50us) raft_consensus.cc:1685] finished
0709 12:10:45.039745 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.039911 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":299365,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":93,"raft.queue_time_us":103827,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":789811,"threads_started":2,"wal-append.queue_time_us":412003}
I20260709 12:10:45.055368 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:45.056069 10585 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:45.059144 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:45.040786 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.041661 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.064643 10433 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:45.042447 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 217171us
I20260709 12:10:45.065904 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:45.065521 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1528) took 1094 ms. Trace:
I20260709 12:10:45.067811 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:45.067889 10510 rpcz_store.cc:276] 0709 12:10:43.971381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.900282 (+928901us) service_pool.cc:224] Handling call
0709 12:10:45.065001 (+164719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:45.065200 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:45.065226 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:45.065267 (+    41us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:45.065307 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.065362 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.065460 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:45.047431 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2003) took 1071 ms. Trace:
I20260709 12:10:45.069752 11178 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:45.070063 13214 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:45.072059 13066 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:45.072624 13066 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:45.073035 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:45.073931 12928 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:45.047307 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:45.048774 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:44.954535 13082 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.075232 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.069629 10506 rpcz_store.cc:276] 0709 12:10:43.975525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.960334 (+984809us) service_pool.cc:224] Handling call
0709 12:10:44.978494 (+ 18160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:45.047072 (+ 68578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:45.047109 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:45.047155 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:45.047194 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.047251 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.047367 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:45.073552 13066 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:44.926658 13184 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.074396 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:45.085978 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.050186 13043 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.092442 10585 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:45.087596 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.061192 13206 raft_consensus.cc:2764] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:10:45.086680 10510 raft_consensus.cc:2376] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 4. Re-sending same reply.
I20260709 12:10:45.096952 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:45.102558 13238 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:45.113139 13238 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.113622 13238 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:45.114740 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.115716 10506 raft_consensus.cc:2376] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7. Re-sending same reply.
I20260709 12:10:44.985721 13213 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.117733 10434 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:45.097582 10583 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:45.078583 12829 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:45.111089 10433 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
W20260709 12:10:44.951498 10333 rpcz_store.cc:269] 0709 12:10:24.851873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.852481 (+   608us) service_pool.cc:224] Handling call
0709 12:10:44.923606 (+20071125us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:37.850565 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:37.896459 (+ 45894us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:37.896517 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:10:37.896550 (+    33us) tablet.cc:662] Decoding operations
0709 12:10:37.905210 (+  8660us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:37.905305 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:37.905331 (+    26us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:10:37.933487 (+ 28156us) tablet.cc:701] Row locks acquired
0709 12:10:37.933556 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:10:37.933755 (+   199us) write_op.cc:270] Start()
0709 12:10:37.933950 (+   195us) write_op.cc:276] Timestamp: P: 1783599037933679 usec, L: 0
0709 12:10:37.933976 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:10:38.074454 (+140478us) log.cc:844] Serialized 7887 byte log entry
0709 12:10:38.839263 (+764809us) op_driver.cc:464] REPLICATION: finished
0709 12:10:39.085096 (+245833us) write_op.cc:301] APPLY: starting
0709 12:10:39.085240 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:39.085729 (+   489us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:39.088755 (+  3026us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:39.105068 (+ 16313us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:39.105492 (+   424us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 12:10:39.105539 (+    47us) write_op.cc:312] APPLY: finished
0709 12:10:39.106058 (+   519us) log.cc:844] Serialized 139 byte log entry
0709 12:10:39.106287 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:39.106497 (+   210us) write_op.cc:454] Released schema lock
0709 12:10:39.106577 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:39.256065 (+149488us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adae28 00007f2c96515581 00007f2c96514d58 00007f2c965199f1 00007f2c965199a9 00007f2c96519939 00007f2c96519901 00007f2c96518b2d 0000000000388634 0000000000388479
0709 12:10:39.256370 (+   305us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:39.256420 (+    50us) write_op.cc:454] Released schema lock
0709 12:10:39.276560 (+ 20140us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adc32a 00007f2c92a61f14 00007f2c92a61ec3 00007f2c92a61e38 00007f2c92a61df3 00007f2c92a610dc 00007f2c92a8ae16 00007f2c92a72cbb 00007f2c92a5d39d 00007f2c92a60d75
Metrics: {"mutex_wait_us":18024,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":245453,"apply.run_cpu_time_us":6289,"apply.run_wall_time_us":21980,"num_ops":20,"prepare.queue_time_us":186750,"prepare.run_cpu_time_us":41690,"prepare.run_wall_time_us":1405823,"raft.queue_time_us":199274,"raft.run_cpu_time_us":20166,"raft.run_wall_time_us":468390,"replication_time_us":891922,"spinlock_wait_cycles":35815936,"thread_start_us":614157,"threads_started":4,"wal-append.queue_time_us":373856,"wal-append.run_cpu_time_us":18672,"wal-append.run_wall_time_us":1643478}]]}
I20260709 12:10:45.083040 13155 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:45.120873 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.121414 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.107290 13139 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.142009 12299 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 12:10:45.139853 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44086.4us
I20260709 12:10:45.084733 13218 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.146466 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.150875 10434 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:45.148128 13230 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:45.149505 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:45.152750 10583 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:45.156639 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.123795 13212 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:45.161453 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 72546.3us
W20260709 12:10:45.156665 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 81506.6us
I20260709 12:10:45.149722 13239 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:45.159370 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.159937 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.163097 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:45.166476 10432 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:45.158334 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.126596 13140 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.165192 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.168499 11219 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:45.169829 10505 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:10:45.143500 13155 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:45.141266 12829 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:45.172111 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.161895 13212 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:45.182446 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.141145 13192 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:10:45.165774 13230 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:45.172274 13043 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:45.191721 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.173458 13239 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:45.178136 13155 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:45.156301 13139 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.195323 10433 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:45.192629 10432 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:45.164870 13125 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.187769 13212 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:45.201506 13242 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:45.200412 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.204814 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.205967 13239 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.210404 10433 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:45.207628 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.181428 13243 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:45.213307 10432 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:45.213758 13184 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.213958 12928 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:45.215503 13082 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.216748 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1525) took 1249 ms. Trace:
I20260709 12:10:45.237377 10511 rpcz_store.cc:276] 0709 12:10:43.968740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.896234 (+927494us) service_pool.cc:224] Handling call
0709 12:10:44.896646 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.897072 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.897111 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.897678 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:10:45.096216 (+198538us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:45.096262 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.096347 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:45.216450 (+120103us) raft_consensus.cc:1685] finished
0709 12:10:45.216483 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.216660 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1273,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":198088,"threads_started":1,"wal-append.queue_time_us":285343}
I20260709 12:10:45.189604 13207 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.196017 13140 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:45.242817 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.206459 12659 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.198870 13169 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:45.207831 13043 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:45.248342 10432 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:45.195880 13206 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:10:45.218180 13243 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.243152 13140 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.241654 13194 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:45.254954 13061 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.594s	user 0.017s	sys 0.000s
I20260709 12:10:45.199632 12829 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:10:45.270850 11216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 346) took 1687 ms (client timeout 1928 ms). Trace:
I20260709 12:10:45.264384 10433 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:45.244725 13218 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:45.266928 13240 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:10:45.267086 13250 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:10:45.268520 13245 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:10:45.272507 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:45.250204 12928 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:45.271409 11216 rpcz_store.cc:269] 0709 12:10:43.583428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.586743 (+  3315us) service_pool.cc:224] Handling call
0709 12:10:45.270583 (+1683840us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400240ed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.270769 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363890432}
I20260709 12:10:45.273891 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1829) took 1585 ms. Trace:
I20260709 12:10:45.282903 10435 rpcz_store.cc:276] 0709 12:10:43.688472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.170529 (+482057us) service_pool.cc:224] Handling call
0709 12:10:44.170936 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.171375 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.171416 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.171960 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.377462 (+205502us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:44.377527 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.377635 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:45.269165 (+891530us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b2400172cb8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa30fb0 00007f2c926a7488 00007f2c926a7be8 00007f2c9267f398 00007f2c9267a76a 00007f2c92679bb6
0709 12:10:45.271992 (+  2827us) raft_consensus.cc:1685] finished
0709 12:10:45.272020 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.273815 (+  1795us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1624,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":5582,"spinlock_wait_cycles":6750464,"thread_start_us":205023,"threads_started":1,"wal-append.queue_time_us":522283}
I20260709 12:10:45.254696 13243 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:45.274560 13061 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.272019 13194 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.280529 13173 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:45.265111 13140 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:45.278663 13043 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.298511 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:45.300376 10581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1818) took 1681 ms (client timeout 2172 ms). Trace:
W20260709 12:10:45.300813 10581 rpcz_store.cc:269] 0709 12:10:43.619948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.620094 (+   146us) service_pool.cc:224] Handling call
0709 12:10:45.300168 (+1680074us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b540010b8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.300305 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478542720}
I20260709 12:10:45.277864 13250 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:45.322036 12928 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:45.324524 13194 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.282686 11217 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:45.345569 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47175.5us
I20260709 12:10:45.287279 13213 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:45.355821 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91690.4us
I20260709 12:10:45.354930 13194 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:45.375384 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1830) took 1666 ms. Trace:
I20260709 12:10:45.257234 13230 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.376062 10437 rpcz_store.cc:276] 0709 12:10:43.708644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.328073 (+619429us) service_pool.cc:224] Handling call
0709 12:10:44.379592 (+ 51519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.379845 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.379871 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.401856 (+ 21985us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.816304 (+414448us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.816345 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.816417 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:45.375042 (+558625us) raft_consensus.cc:1685] finished
0709 12:10:45.375076 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.375292 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":361112,"threads_started":1,"wal-append.queue_time_us":649558}
I20260709 12:10:45.384019 11216 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:45.385740 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.387348 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.388983 10435 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:45.389391 13206 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.389977 13206 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.391788 13139 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:45.393028 13200 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.222s	user 0.004s	sys 0.001s
I20260709 12:10:45.392982 13206 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.394780 13180 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:45.395967 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:45.369805 13145 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.005s
I20260709 12:10:45.346899 13198 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.378890 13125 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.406040 13258 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:45.415642 13258 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.416220 13258 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:45.397974 13177 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.000s
I20260709 12:10:45.406000 13139 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.419198 13177 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.419967 13139 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.421026 10584 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:45.419960 13061 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.356031 13250 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.423942 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.372445 13251 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.432118 10437 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:45.434245 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.435254 11216 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:10:45.437243 10582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2043) took 1485 ms (client timeout 1796 ms). Trace:
I20260709 12:10:45.406569 13145 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:45.437664 10582 rpcz_store.cc:269] 0709 12:10:43.951562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.969364 (+ 17802us) service_pool.cc:224] Handling call
0709 12:10:45.437067 (+1467703us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b540010c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.437175 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1368307584}
W20260709 12:10:45.439271 13261 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:10:45.441200 13233 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.545s	user 0.000s	sys 0.001s
I20260709 12:10:45.441327 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 217) took 1787 ms. Trace:
I20260709 12:10:45.441823 10508 rpcz_store.cc:276] 0709 12:10:43.653919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.707695 (+ 53776us) service_pool.cc:224] Handling call
0709 12:10:44.250463 (+542768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.251258 (+   795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.440922 (+1189664us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54002f2bd0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.441231 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213316352}
I20260709 12:10:45.447899 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1832) took 1747 ms. Trace:
I20260709 12:10:45.448624 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.449086 10510 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:10:45.449924 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.450678 10510 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 3
I20260709 12:10:45.451010 10510 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260709 12:10:45.448359 10587 rpcz_store.cc:276] 0709 12:10:43.700541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.700668 (+   127us) service_pool.cc:224] Handling call
0709 12:10:43.701008 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.701460 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:43.857911 (+156451us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18003c8c20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c964d73c7 00007f2c92ae99a7 00007f2c92af90bf 00007f2c92af9038
0709 12:10:44.317728 (+459817us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.821855 (+504127us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.879297 (+ 57442us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.879539 (+   242us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:44.879595 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.879708 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:45.439713 (+560005us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:45.447658 (+  7945us) raft_consensus.cc:1685] finished
0709 12:10:45.447684 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:45.447830 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":141910,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":6093,"raft.queue_time_us":97005,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":11040,"spinlock_wait_cycles":15900416,"thread_start_us":884876,"threads_started":3,"wal-append.queue_time_us":526761}
I20260709 12:10:45.423724 13250 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:45.422410 13140 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.459460 10510 raft_consensus.cc:2917] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260709 12:10:45.461268 12945 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.460146 12659 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.461987 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:45.460072 11213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 392) took 1789 ms (client timeout 1933 ms). Trace:
W20260709 12:10:45.461114 13122 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.688s	user 0.006s	sys 0.006s
I20260709 12:10:45.464488 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.465262 13238 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.471210 13177 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.471942 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.470268 10510 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:45.480187 13255 consensus_queue.cc:1175] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "99d42e7288f145f1bd5af212fd062995" responder_term: 7 status { last_received { term: 4 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: [6.3-6.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:10:45.461532 13145 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.481999 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.483387 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.484750 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.484894 13212 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.487084 13150 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:45.488976 13198 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.493283 13071 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:45.496047 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.499435 13177 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.499756 10511 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:45.499963 13156 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.500244 13177 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.456202 13251 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.504952 13177 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.505894 13251 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.479192 12945 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.425288 13206 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:10:45.474802 11213 rpcz_store.cc:269] 0709 12:10:43.668315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.677196 (+  8881us) service_pool.cc:224] Handling call
0709 12:10:44.142178 (+464982us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:45.442890 (+1300712us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b54004be454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.457974 (+ 15084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374592512}
I20260709 12:10:45.470083 13139 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.509470 13238 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.512060 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.512944 11216 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:45.514998 13122 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.515527 13155 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.518167 13122 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.519251 13155 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:45.521325 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.521639 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.522083 13070 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.522377 11213 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:45.525194 10510 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:45.518934 13122 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:45.526533 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:45.527429 10431 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:45.529212 10435 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:45.526168 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:45.533875 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.535362 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.540504 10508 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:45.509464 12945 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.544059 13238 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.545058 13177 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:45.547278 13177 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.546231 12829 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:45.549331 12829 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.549953 12829 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:45.550302 11216 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:45.533286 13140 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.548525 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:45.551328 13070 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:45.551995 11213 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:45.552969 13140 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:45.555334 13198 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:45.555794 10584 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:45.558421 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:45.544879 13238 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.559350 11217 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:45.566089 13212 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:45.567282 13265 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
I20260709 12:10:45.578905 10582 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:45.581501 10585 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:45.583031 10587 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:45.584926 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.587280 10587 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:45.533811 13122 consensus_queue.cc:1243] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:10:45.588629 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:45.590559 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:45.594457 10579 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:45.591076 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:45.592314 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.598037 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.600392 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.603286 10506 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:45.605296 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:45.650094 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44127us
I20260709 12:10:45.652742 10579 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:45.607537 13251 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:10:45.676431 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.674062 10579 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:45.760492 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.752893 12299 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:45.760788 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 84447.5us
I20260709 12:10:45.761567 12299 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.649574 13218 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:45.716717 13267 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:10:45.763213 13140 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:45.709892 13214 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:45.717442 13176 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:45.763882 13140 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.715721 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.761315 10506 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:45.761893 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:45.769398 13194 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:45.682825 13189 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:45.770351 13176 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.771181 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.607734 13044 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:45.662211 10508 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:45.680801 10587 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:45.670735 13193 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.773102 13169 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:45.764154 13214 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.778028 13070 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:45.709654 13122 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:10:45.791788 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:45.713526 13177 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:45.783844 13044 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:45.800421 10431 raft_consensus.cc:2368] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4.
I20260709 12:10:45.806054 10431 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:45.788497 13169 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.809883 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.811208 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.675354 13212 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.127s
I20260709 12:10:45.792779 13070 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:45.782617 13189 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.796584 13258 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.999858 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.858219 10513 raft_consensus.cc:2410] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:46.001539 11212 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:10:46.002053 10513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2005) took 2025 ms (client timeout 1773 ms). Trace:
I20260709 12:10:45.820384 13169 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:45.858633 13140 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:45.831530 13243 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.534s	user 0.003s	sys 0.012s
I20260709 12:10:45.823035 13259 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:45.861837 13177 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:45.865044 10504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2006) took 1888 ms (client timeout 1548 ms). Trace:
I20260709 12:10:45.826565 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.867967 13070 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:45.833007 13234 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:10:45.826876 13238 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:45.874913 10430 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 1821) took 2206 ms (client timeout 1905 ms). Trace:
I20260709 12:10:45.875617 13044 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:45.877887 12829 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:45.879134 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:45.879951 10436 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:10:45.881033 11213 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:45.885109 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:46.047760 11221 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:45.887370 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2011) took 1902 ms. Trace:
W20260709 12:10:46.039327 13240 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.178s	user 0.004s	sys 0.000s
I20260709 12:10:45.893642 10507 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
W20260709 12:10:45.908396 10512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2008) took 1931 ms (client timeout 2572 ms). Trace:
W20260709 12:10:46.052136 10512 rpcz_store.cc:269] 0709 12:10:43.977072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.747923 (+770851us) service_pool.cc:224] Handling call
0709 12:10:45.908113 (+1160190us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b540036ef54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.908304 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184151296}
I20260709 12:10:45.911321 12299 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.912636 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:45.919520 13270 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:10:45.928894 13125 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:45.933751 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.944096 13237 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:45.945560 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:45.946386 12866 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:45.947865 10579 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:46.063519 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:45.948601 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:45.950169 13139 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:45.951431 13176 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:45.967001 12659 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:45.975548 13150 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:46.047570 10322 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:45.975831 13254 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:45.980266 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.082036 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:45.984553 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:45.989356 10580 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:45.992626 10583 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.084198 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.085163 10583 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:45.999476 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.085933 10587 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:46.000008 11217 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
W20260709 12:10:45.859680 13274 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:10:46.066973 13237 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:10:46.002485 10513 rpcz_store.cc:269] 0709 12:10:43.976140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.367028 (+390888us) service_pool.cc:224] Handling call
0709 12:10:46.001826 (+1634798us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400228dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:46.001967 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3340550528}
I20260709 12:10:46.013962 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:46.021076 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.035427 10504 rpcz_store.cc:269] 0709 12:10:43.976454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.997109 (+ 20655us) service_pool.cc:224] Handling call
0709 12:10:45.841599 (+1844490us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54000f96d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.864941 (+ 23342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351722880}
W20260709 12:10:46.100428 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 86622.1us
I20260709 12:10:46.100543 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.038793 11218 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 LEARNER]: Advancing to term 7
I20260709 12:10:46.040537 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.002638 13258 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:46.043255 10430 rpcz_store.cc:269] 0709 12:10:43.668773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.690775 (+ 22002us) service_pool.cc:224] Handling call
0709 12:10:45.874625 (+2183850us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54003534d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.874808 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689444608}
I20260709 12:10:46.002737 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.045647 10510 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:46.003173 13155 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:46.048599 10505 rpcz_store.cc:276] 0709 12:10:43.984681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:45.046502 (+1061821us) service_pool.cc:224] Handling call
0709 12:10:45.046871 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:45.170235 (+123364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.887049 (+716814us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400412654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:45.887287 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282129152}
I20260709 12:10:46.038034 13259 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:45.893443 10578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1825) took 2270 ms (client timeout 2498 ms). Trace:
W20260709 12:10:46.104951 10578 rpcz_store.cc:269] 0709 12:10:43.622575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.626131 (+  3556us) service_pool.cc:224] Handling call
0709 12:10:45.893121 (+2266990us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54001d56d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:45.893308 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661017984}
I20260709 12:10:46.106597 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:46.051074 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 250411us
W20260709 12:10:46.110705 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.112753 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.055490 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.068449 12659 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:46.073012 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.115067 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:46.079262 10322 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45327.2us
I20260709 12:10:46.034024 13243 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.060514 13214 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.089440 11219 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.129004 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.043757 13177 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:45.990919 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.087630 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:46.094110 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:46.098385 13156 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:46.136152 10583 raft_consensus.cc:2368] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4.
I20260709 12:10:46.136277 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.092569 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.137713 10580 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:46.096119 10508 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:46.138473 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:46.139101 10508 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:46.139256 11220 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:46.096767 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:46.140357 10582 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:46.097991 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:46.070922 12866 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:46.102500 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.073769 12299 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:46.104552 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.081094 13254 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:46.119289 11177 connection.cc:256] RPC call timeout handler was delayed by -1.0632557670001006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 12:10:46.128144 11215 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:46.128808 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:46.145865 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 155068us
I20260709 12:10:46.146095 10438 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.146597 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.129927 10579 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:46.131742 10430 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:46.133069 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.146936 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.153420 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.133504 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.137863 10513 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.154285 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:46.142915 10505 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:46.104697 13237 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.157582 10436 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:46.083773 13239 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:10:46.114295 13125 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:46.143462 10578 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:46.159833 10586 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:46.161690 10511 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:46.155603 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.083393 13150 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.132819 13258 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:46.172667 10513 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:46.167367 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.167819 10583 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:46.168992 10579 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:46.145131 13098 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.144767 13212 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.148375 13214 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:46.180989 10504 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:46.164459 13235 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:46.183686 10511 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:46.117038 13155 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.144932 13193 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.167444 13205 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:46.190528 11218 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:46.193729 13212 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.188827 13098 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.198455 13212 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.199021 13098 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.199276 13193 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.201635 13193 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.202893 10479 leader_election.cc:400] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4.
I20260709 12:10:46.203248 10479 leader_election.cc:403] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:10:46.203387 10507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2002) took 2228 ms (client timeout 1553 ms). Trace:
W20260709 12:10:46.203788 10507 rpcz_store.cc:269] 0709 12:10:43.975181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.287031 (+311850us) service_pool.cc:224] Handling call
0709 12:10:46.203184 (+1916153us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000daf54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:46.203311 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454948224}
I20260709 12:10:46.205013 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.206521 10507 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:10:46.208137 13150 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:46.174276 13239 consensus_queue.cc:1243] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:10:46.172969 13236 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:46.172133 13227 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:46.231555 10504 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:46.237560 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1481) took 1089 ms. Trace:
W20260709 12:10:46.237740 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 95078us
I20260709 12:10:46.240617 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.242022 13214 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:46.242691 13214 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:46.160148 13156 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:46.255406 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 65963.7us
I20260709 12:10:46.238158 10432 rpcz_store.cc:276] 0709 12:10:45.148625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:45.249896 (+101271us) service_pool.cc:224] Handling call
0709 12:10:45.251784 (+  1888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:45.252344 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:45.258686 (+  6342us) log.cc:844] Serialized 14 byte log entry
0709 12:10:45.508662 (+249976us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b180009b080. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f23a 00007f2c92ae9a2a
0709 12:10:45.659971 (+151309us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:45.710275 (+ 50304us) log.cc:844] Serialized 26 byte log entry
0709 12:10:45.710504 (+   229us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:45.710554 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.710651 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.229361 (+518710us) raft_consensus.cc:1685] finished
0709 12:10:46.237346 (+  7985us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.237484 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5818,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":4851,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":8673,"spinlock_wait_cycles":5014400,"thread_start_us":150976,"threads_started":1,"wal-append.queue_time_us":494426}
I20260709 12:10:46.257977 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.193764 13235 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:46.193343 10587 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:46.242141 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:46.290114 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48088.4us
I20260709 12:10:46.179472 13209 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 12:10:46.170519 13237 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.184092 13192 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
W20260709 12:10:46.283680 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 80983us
I20260709 12:10:46.228509 13227 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:46.285856 10438 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:46.228659 13258 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:10:46.279569 13168 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:10:46.226161 13218 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:10:46.283648 13235 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.243948 13214 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.283725 13205 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:46.289196 13251 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4.
I20260709 12:10:46.260437 13211 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:10:46.261415 12706 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:10:46.294328 13140 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:46.295948 13150 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:46.296175 12829 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.300845 13227 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:46.331327 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:46.315917 12866 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:46.324167 13273 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:10:46.318719 13189 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.322077 13098 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:46.358482 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65958.4us
I20260709 12:10:46.340512 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:46.362644 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.363488 10578 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:46.327280 13193 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:10:46.336268 13285 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:46.337639 13150 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.361855 13243 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.364595 13259 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.387254 13290 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.370030 13251 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:46.425125 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93688.1us
W20260709 12:10:46.386816 13278 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:10:46.389362 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:46.402568 12664 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:46.371030 13286 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:46.433579 10512 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:10:46.437260 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:46.374481 13177 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:46.359526 13098 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.358976 13180 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 12:10:46.428859 13281 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:10:46.349138 12866 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:46.416236 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96135.8us
W20260709 12:10:46.449923 13287 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:10:46.449090 13297 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.442673 13286 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.459944 13218 consensus_queue.cc:1243] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:46.467813 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:46.400640 13285 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.452941 13155 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:46.467371 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78097.4us
I20260709 12:10:46.475046 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.469925 10433 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:46.477234 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:46.469221 13277 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:10:46.478065 10438 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:46.474661 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 216778us
I20260709 12:10:46.485504 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:46.486618 10432 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:46.492165 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.492815 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:46.444280 13290 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:46.495527 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.448372 12664 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.470633 13125 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:46.496846 11219 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:46.469003 13286 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:46.443446 13205 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.511710 13236 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:46.512310 13236 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:46.504184 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:46.513262 13236 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.514427 10436 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:46.514380 13236 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:46.511843 10582 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:46.505306 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.516101 13173 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:46.516563 13173 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:46.517644 13173 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.520478 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.521148 10508 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:46.481247 13237 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.525727 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:46.496434 13238 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:46.553830 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:46.555281 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.555603 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:46.556250 10511 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:46.556471 10510 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:46.560272 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.533290 10508 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:46.534112 12299 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:46.565994 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.480638 13251 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:10:46.549968 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:46.501972 13155 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:46.528776 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.572084 12299 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:46.573585 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:46.576300 10512 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
W20260709 12:10:46.577066 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.523499 13098 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.507431 13125 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.581151 13295 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:46.582242 13294 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:10:46.582000 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:46.582818 13168 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.583644 13150 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:46.583771 13206 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:46.576691 13238 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:46.584914 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:46.545688 13189 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:10:46.502445 13140 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:10:46.617609 13238 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.563952 13211 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:46.564065 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.507836 13193 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:10:46.621194 13193 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:46.621598 13193 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:46.605685 13168 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:46.623875 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 183706us
W20260709 12:10:46.597920 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 206516us
I20260709 12:10:46.550657 13194 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:10:46.547937 13218 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:10:46.545101 13177 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:10:46.472082 13297 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.578312 13155 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:46.600776 12299 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.567603 13212 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:10:46.594544 13125 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:46.612586 10478 leader_election.cc:400] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 3 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 3. Current term is 4.
I20260709 12:10:46.649107 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:46.614229 13290 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:46.614976 12866 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.615041 13286 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.615572 13305 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:10:46.616753 13070 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.000s
I20260709 12:10:46.616957 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.566781 13304 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:10:46.604182 13156 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.634099 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:46.603592 13098 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:46.669242 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:46.703377 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:46.704525 10508 raft_consensus.cc:606] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:46.705008 10508 raft_consensus.cc:614] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:46.676162 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:46.715503 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:46.740360 10585 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:46.742082 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:46.761871 10581 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:46.787358 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.801311 10585 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.639513 13308 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:46.851922 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 136523us
I20260709 12:10:46.856925 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.873842 10578 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:46.858767 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.622601 13193 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.876617 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:46.656085 13302 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:46.655056 13139 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:10:46.668246 13290 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.899034 10586 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:46.672899 13150 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:46.675338 13305 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:46.935056 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:46.653859 13285 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.887408 11217 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:46.879580 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:46.654028 13212 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:46.656426 13286 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:46.888149 13168 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.652215 13297 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.653435 12866 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.654632 12706 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.667668 13098 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.656476 13070 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.946189 10587 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:46.881891 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:46.651420 13295 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:46.891258 13302 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:46.938599 13139 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:46.968123 13139 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.968365 10510 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:46.923118 13150 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.962968 13324 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.969309 13150 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:46.968657 13156 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:46.969763 13315 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 12:10:46.970470 13303 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:46.971095 13303 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.969669 13324 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.973714 13303 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.974028 13324 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.944851 13044 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:46.975462 13312 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 12:10:46.947072 13193 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:46.962253 13297 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:46.959548 12706 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:46.986068 12706 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:46.997083 13070 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:46.998303 10509 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:10:47.000000 13150 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.000931 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:46.892254 13308 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.001597 10435 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.002389 13150 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:47.003577 13308 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:47.004875 10513 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:47.012699 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:47.015547 13259 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:46.926062 13290 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:46.959663 13098 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:46.968590 13139 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:46.923396 13305 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.884858 13310 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 12:10:46.944700 13212 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:46.950898 13285 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:46.966763 13319 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:10:47.030071 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:47.040300 10435 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:47.041105 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:47.041814 10510 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:47.042948 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:47.043744 10510 raft_consensus.cc:2376] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7. Re-sending same reply.
I20260709 12:10:46.957501 12866 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.043938 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:47.036437 13285 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.034674 13305 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:47.078718 13306 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:47.081041 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1858) took 1081 ms. Trace:
I20260709 12:10:47.082566 13150 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:47.085109 13206 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:46.955833 13155 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.026378 13259 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:10:47.084185 13335 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:47.062669 13230 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:47.034327 13208 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:10:47.095927 10582 rpcz_store.cc:276] 0709 12:10:45.999609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.167092 (+167483us) service_pool.cc:224] Handling call
0709 12:10:46.511625 (+344533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:46.515380 (+  3755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:47.080723 (+565343us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54001a13d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:47.080944 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783168896}
I20260709 12:10:47.069537 13308 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.116568 10432 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:47.092172 13285 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:47.136303 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.143630 13308 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.104414 13335 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:47.104234 13230 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.103618 13150 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.112670 13206 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.220461 12866 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:47.221987 13206 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.224547 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:47.225520 10432 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 3.
W20260709 12:10:47.209304 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.095686 13306 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:47.115278 13302 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.227639 13306 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.228549 13302 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.228716 13306 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:47.230362 13236 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.011s
I20260709 12:10:47.212369 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:47.086349 12659 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.231534 12659 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:47.233872 13235 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.905s	user 0.011s	sys 0.001s
I20260709 12:10:47.234490 12664 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:47.234337 13235 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.214226 13308 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:47.210613 13230 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.214779 13254 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.311864 13254 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.216739 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:47.315330 13235 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:47.208498 13335 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.317495 13290 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.118785 13295 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.100816 13209 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:47.323405 13209 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.218299 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1557) took 1200 ms. Trace:
I20260709 12:10:47.327219 10513 rpcz_store.cc:276] 0709 12:10:46.018093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.173753 (+155660us) service_pool.cc:224] Handling call
0709 12:10:46.174058 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:47.005248 (+831190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:47.217998 (+212750us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54003fd9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:47.218227 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909854080}
I20260709 12:10:47.219353 13150 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:47.038429 13212 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:47.237444 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.329481 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.329945 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.241186 13211 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:47.332599 13211 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:47.333983 13211 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.243618 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:47.243716 10579 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:10:47.354357 10579 pending_rounds.cc:85] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 6
I20260709 12:10:47.354712 10579 pending_rounds.cc:107] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:10:47.247587 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:47.336877 13193 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.337594 13242 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.214093 11218 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:10:47.430420 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.017s	sys 0.073s
W20260709 12:10:47.430770 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.017s	sys 0.073s
I20260709 12:10:47.339030 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:47.431378 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 554901us
I20260709 12:10:47.364313 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:47.432083 10511 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:47.376178 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 128714us
W20260709 12:10:47.383522 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.418416 13211 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.421334 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:47.437633 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:47.207722 13309 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.021s	sys 0.033s
W20260709 12:10:47.428519 13169 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.394s	user 0.012s	sys 0.005s
I20260709 12:10:47.443564 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:47.427340 10505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.032s	sys 0.090s
W20260709 12:10:47.428783 10322 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:47.354960 10579 raft_consensus.cc:2889] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:47.445771 10505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.032s	sys 0.090s
I20260709 12:10:47.431573 10506 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
W20260709 12:10:47.449198 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.438589 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 394738us
I20260709 12:10:47.429054 13290 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:47.429270 13239 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:47.433643 13308 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:47.439467 13309 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.021s	sys 0.034s
I20260709 12:10:47.492120 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:47.433480 13209 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:47.449893 13205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.016s	sys 0.020s
I20260709 12:10:47.493371 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:47.494086 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:47.501935 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.510469 10584 raft_consensus.cc:2376] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6. Re-sending same reply.
I20260709 12:10:47.518772 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:47.434165 13193 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.449306 12659 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 12:10:47.500430 13206 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.451020 13169 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:47.515450 13205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.016s	sys 0.021s
I20260709 12:10:47.440269 13305 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.439349 13242 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:47.527946 12664 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:47.530501 13098 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.493321 13333 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.517616 13308 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:47.596501 13169 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:47.596717 13193 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.510452 13290 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.512264 13239 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.599949 13239 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:47.579306 12664 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:47.600910 12664 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.602052 13290 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:47.587642 13323 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:10:47.589926 13305 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:47.592864 13227 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.243s	user 0.018s	sys 0.004s
I20260709 12:10:47.607504 13227 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:47.594235 13125 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.921s	user 0.019s	sys 0.000s
I20260709 12:10:47.611593 13125 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.518153 13235 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:10:47.516194 13139 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.570945 13206 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.600149 13242 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.590593 13098 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.618633 13227 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.597112 13308 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.578285 13335 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:47.633553 13255 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:47.635511 13255 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:47.652401 13285 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:47.699152 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 178293us
W20260709 12:10:47.701920 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.706146 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:47.708402 10506 raft_consensus.cc:2368] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7.
I20260709 12:10:47.716385 13098 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.716684 12659 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:47.624406 13125 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:47.727862 13155 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:47.730737 11218 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:47.808301 13140 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:47.817010 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.796083 13176 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:47.749126 10509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2030) took 1751 ms (client timeout 2233 ms). Trace:
W20260709 12:10:47.924782 13254 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.612s	user 0.017s	sys 0.003s
W20260709 12:10:47.700049 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 256336us
I20260709 12:10:47.949373 13285 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:47.808852 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.960726 12659 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:47.961688 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.915417 13255 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.909101 13212 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.765374 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:47.761788 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 345) took 1712 ms. Trace:
I20260709 12:10:47.962939 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:47.798853 10511 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:47.794144 12664 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.764427 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:47.725190 12866 raft_consensus.cc:2764] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:10:47.955120 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.809787 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.938339 13258 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:48.016987 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.956970 13139 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:47.815452 13192 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:47.783495 12945 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:48.033975 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.953118 13173 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.906270 13206 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.009702 10579 rpcz_store.cc:276] 0709 12:10:46.049034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.184401 (+135367us) service_pool.cc:224] Handling call
0709 12:10:46.185699 (+  1298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:47.446147 (+1260448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:47.761095 (+314948us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002779d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:47.761500 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432163072}
I20260709 12:10:47.736140 10336 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 4, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 4 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:48.032431 12945 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:47.957506 10509 rpcz_store.cc:269] 0709 12:10:45.997501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.081740 (+ 84239us) service_pool.cc:224] Handling call
0709 12:10:47.748536 (+1666796us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54000d9b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:47.748859 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105190656}
I20260709 12:10:48.016987 13258 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:47.953888 13140 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:47.956012 13176 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.023798 13192 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:48.009654 13098 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:48.045676 11217 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:48.058729 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.059597 13239 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:48.059928 13336 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:48.060410 13239 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.062815 12664 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.031697 13155 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:48.077069 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.062943 13255 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.081219 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:48.086028 13304 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:48.087188 13304 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.086032 13150 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:48.091117 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 128475us
I20260709 12:10:48.091326 13150 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.092470 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.094123 13150 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.068670 13239 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.103858 12945 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.105443 12945 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:48.108871 13335 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.110422 13139 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:48.115985 13230 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.808s	user 0.014s	sys 0.012s
I20260709 12:10:48.116686 13230 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:48.108963 13325 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.882s	user 0.003s	sys 0.003s
I20260709 12:10:48.119449 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.063747 13285 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:10:48.095125 13214 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:48.101786 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 487111us
I20260709 12:10:48.090420 13304 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.155936 13176 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.157186 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.159924 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:48.172196 13336 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:48.204830 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.200742 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:48.139048 12866 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:10:48.201151 13230 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:48.223907 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.230975 10509 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:48.233404 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:48.233547 13098 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:10:48.228937 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:48.241394 13307 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:48.244944 11215 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
W20260709 12:10:48.247143 13211 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.754s	user 0.003s	sys 0.022s
I20260709 12:10:48.247843 13290 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:48.248340 13290 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:48.247886 13237 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:10:48.255089 13309 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:48.257324 13243 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:48.258121 13243 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.260684 13243 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.262257 13155 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.226285 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:48.216435 13304 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:48.284888 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 127898us
W20260709 12:10:48.287441 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.294607 10506 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:10:48.294754 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0881081859999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 12:10:48.295374 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0571815020000486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 12:10:48.296840 13177 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.297674 13177 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.299193 13239 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:48.300004 13239 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.302374 13239 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.195389 13258 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.303719 13140 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:48.305140 13299 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 1.387s	user 0.000s	sys 0.003s
I20260709 12:10:48.221055 13214 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.306561 13206 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.307278 13206 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.308519 13177 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.308718 13214 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.309537 13206 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.310827 10336 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 6, now has a pending config: VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67), VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 6 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:10:48.314469 13327 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.097s	user 0.000s	sys 0.003s
W20260709 12:10:48.316747 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.316996 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 231) took 1688 ms. Trace:
W20260709 12:10:48.317243 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.317809 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.317462 13304 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.319351 12534 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:48.319823 13292 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.056s	user 0.001s	sys 0.002s
I20260709 12:10:48.319990 12534 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:48.320614 13071 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.320837 13304 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.302882 13239 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.314078 10336 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:10:48.299599 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45346.9us
W20260709 12:10:48.349179 13352 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:10:48.354753 10509 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:48.350448 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:48.359124 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:48.359653 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:48.360069 10432 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:48.360652 10433 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.339500 12534 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.321405 13071 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.363314 12659 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:48.363829 13343 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:10:48.364176 13071 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.317430 10507 rpcz_store.cc:276] 0709 12:10:46.628450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.628604 (+   154us) service_pool.cc:224] Handling call
0709 12:10:46.649244 (+ 20640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:46.657361 (+  8117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:46.657401 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:46.657922 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:10:47.016338 (+358416us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:47.021417 (+  5079us) log.cc:844] Serialized 14 byte log entry
0709 12:10:47.031267 (+  9850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:47.031381 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:48.313443 (+1282062us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:48.316441 (+  2998us) raft_consensus.cc:1685] finished
0709 12:10:48.316561 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:48.316770 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":233623,"prepare.queue_time_us":10111,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2807,"raft.queue_time_us":21302,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":15168,"spinlock_wait_cycles":96360192,"thread_start_us":128927,"threads_started":1,"wal-append.queue_time_us":413764}
I20260709 12:10:48.365597 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:48.366667 13346 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:10:48.368265 13305 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.341897 13214 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.368965 13305 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.369592 13214 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.371138 13239 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.372702 13214 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.373390 13239 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.378441 13258 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:48.379541 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1570) took 1013 ms. Trace:
I20260709 12:10:48.380563 13290 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.381455 13335 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:48.382072 13335 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:48.382573 13290 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.383545 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1865) took 1585 ms. Trace:
I20260709 12:10:48.383322 13335 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.383929 10583 rpcz_store.cc:276] 0709 12:10:46.755301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.793666 (+ 38365us) service_pool.cc:224] Handling call
0709 12:10:46.878698 (+ 85032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:46.879072 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:48.137091 (+1258019us) log.cc:844] Serialized 14 byte log entry
0709 12:10:48.210213 (+ 73122us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:48.215623 (+  5410us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:48.230209 (+ 14586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.290856 (+ 60647us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:48.340184 (+ 49328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1268556,"prepare.queue_time_us":1339461,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":4107,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":54959,"thread_start_us":44758,"threads_started":1}
I20260709 12:10:48.387601 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1569) took 1022 ms. Trace:
I20260709 12:10:48.388357 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:48.388166 13307 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 3, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.389189 10429 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:48.390657 13176 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:10:48.395083 13348 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:10:48.395417 13302 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:48.396088 13302 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.365484 13301 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.359730 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:48.404407 10434 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:48.405020 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.405637 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.352810 13285 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.359144 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:48.413270 12659 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:48.414137 12659 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:48.416849 12659 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.417538 13243 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.379990 10505 rpcz_store.cc:276] 0709 12:10:47.366223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.462601 (+ 96378us) service_pool.cc:224] Handling call
0709 12:10:47.483973 (+ 21372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:47.499876 (+ 15903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:47.499956 (+    80us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:47.508960 (+  9004us) log.cc:844] Serialized 26 byte log entry
0709 12:10:47.701291 (+192331us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:47.701489 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:47.701681 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:48.379078 (+677397us) raft_consensus.cc:1685] finished
0709 12:10:48.379102 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:48.379306 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13095,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":209,"thread_start_us":191414,"threads_started":1,"wal-append.queue_time_us":652284}
I20260709 12:10:48.421326 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.424739 13285 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.424490 10336 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:10:48.426708 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.426374 10336 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 5, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 5 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:48.428277 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:48.428995 10579 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.431592 10582 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:48.432905 10587 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:48.434350 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.436234 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:48.436880 10513 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.283885 13192 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.431453 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.420424 10432 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:48.445096 10579 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:48.448122 10583 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:48.435832 10336 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:10:48.455237 13358 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:10:48.457778 13302 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:48.458555 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45977.2us
I20260709 12:10:48.465433 13207 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:48.466446 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:48.467316 10505 raft_consensus.cc:606] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:48.467751 10505 raft_consensus.cc:614] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:48.473978 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:48.474716 10585 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.475296 10434 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:48.480302 10513 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:48.474014 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.388100 10512 rpcz_store.cc:276] 0709 12:10:47.365681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.365849 (+   168us) service_pool.cc:224] Handling call
0709 12:10:47.435520 (+ 69671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:47.709953 (+274433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:47.710024 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:47.735416 (+ 25392us) log.cc:844] Serialized 26 byte log entry
0709 12:10:47.998963 (+263547us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:47.999883 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.120774 (+120891us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400229cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:48.120998 (+   224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:48.387074 (+266076us) raft_consensus.cc:1685] finished
0709 12:10:48.387093 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:48.387364 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11545,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":207,"spinlock_wait_cycles":628288768,"thread_start_us":262840,"threads_started":1,"wal-append.queue_time_us":413833}
I20260709 12:10:48.494513 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:48.479518 10336 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:10:48.505478 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.508409 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 184830us
I20260709 12:10:48.506489 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:48.511190 13070 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.511861 13070 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:48.514143 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.516160 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 197160us
I20260709 12:10:48.504238 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:48.526557 10505 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.528940 10587 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 12:10:48.531251 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 173398us
I20260709 12:10:48.537215 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:48.539624 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:48.540474 10582 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.540586 13305 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.525569 13173 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.546473 13173 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.517202 13285 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.548858 13173 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.550171 11216 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:48.535313 10336 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 7, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 7 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:48.496378 13301 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.555776 13301 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.558409 13295 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:48.560585 13295 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:48.561786 13308 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:48.539534 13251 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 12:10:48.564425 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.565025 13251 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:48.565152 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:48.565482 13251 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:48.565909 10508 raft_consensus.cc:606] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:48.566318 10508 raft_consensus.cc:614] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:48.530421 11217 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:48.567672 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.569340 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:48.570226 13212 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:48.571240 13243 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:48.575174 13333 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.932s	user 0.012s	sys 0.004s
I20260709 12:10:48.592041 10505 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:48.597700 10433 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:48.598474 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.599336 10578 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:48.603734 10509 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:48.607414 10511 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:48.622520 13283 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.627568 13122 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:48.629951 13354 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:10:48.595682 13155 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:48.605619 13333 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:48.637166 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:48.639128 13355 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:10:48.640734 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.641176 13070 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.641711 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.642518 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.642005 10336 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 3 to 5, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 5 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:48.643208 10579 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:48.652940 10585 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
W20260709 12:10:48.658334 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 90862.8us
I20260709 12:10:48.659847 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.661517 10582 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:48.566619 13251 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.607303 13139 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:48.666615 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.680188 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:48.684242 10583 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:48.685891 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:48.686159 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.686658 10511 raft_consensus.cc:606] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:10:48.687124 10511 raft_consensus.cc:614] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:48.687933 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.688825 10578 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:48.690011 13207 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:48.691308 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.685822 13122 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.693473 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.694451 13122 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.695240 13070 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:48.695408 12945 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:48.695905 13070 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:48.696010 12945 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:48.696794 12945 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:48.697000 13070 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:48.701086 13359 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.173s	user 0.002s	sys 0.000s
W20260709 12:10:48.705945 13357 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 12:10:48.686055 10336 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 1 to 6, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 6 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:48.711450  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:48.712483 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:48.717120 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:48.719132 13255 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.616s	user 0.015s	sys 0.000s
I20260709 12:10:48.721356 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.722085 13283 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:48.735075 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.806720 10585 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:10:48.726668 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46642.9us
I20260709 12:10:48.736825 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.753345 13212 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:48.758857 10332 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 234) took 19961 ms (client timeout 15000 ms). Trace:
I20260709 12:10:48.683194 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.681912 10581 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:48.811863 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116852us
I20260709 12:10:48.759666 13345 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:48.760512 10587 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:48.761160 13176 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:48.761809 13155 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.762130 13251 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:48.785012  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:48.761191 13192 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:48.802346 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:48.750824 13139 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:48.809494 10580 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:48.814234 10583 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:48.818902 13330 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.818504 13155 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.804855 13207 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.813764 13345 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:48.828418 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 246763us
I20260709 12:10:48.836253 13333 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:48.815397 13212 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.833016 13122 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.831890 13176 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:48.842278 13207 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:48.847559 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 164506us
W20260709 12:10:48.811909 10332 rpcz_store.cc:269] 0709 12:10:28.797335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.797752 (+   417us) service_pool.cc:224] Handling call
0709 12:10:48.758527 (+19960775us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:45.172126 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:45.205735 (+ 33609us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:45.205783 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:10:45.205805 (+    22us) tablet.cc:662] Decoding operations
0709 12:10:45.206799 (+   994us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:45.206891 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:45.206924 (+    33us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:10:45.214291 (+  7367us) tablet.cc:701] Row locks acquired
0709 12:10:45.214313 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:10:45.214505 (+   192us) write_op.cc:270] Start()
0709 12:10:45.214635 (+   130us) write_op.cc:276] Timestamp: P: 1783599045214432 usec, L: 0
0709 12:10:45.214676 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:10:45.215559 (+   883us) log.cc:844] Serialized 3190 byte log entry
0709 12:10:45.364843 (+149284us) op_driver.cc:464] REPLICATION: finished
0709 12:10:45.366522 (+  1679us) write_op.cc:301] APPLY: starting
0709 12:10:45.366626 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:45.366783 (+   157us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:45.366889 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:45.463427 (+ 96538us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:45.463811 (+   384us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:10:45.463909 (+    98us) write_op.cc:312] APPLY: finished
0709 12:10:45.464378 (+   469us) log.cc:844] Serialized 65 byte log entry
0709 12:10:45.464630 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:45.464815 (+   185us) write_op.cc:454] Released schema lock
0709 12:10:45.464893 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:45.508276 (+ 43383us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:45.508341 (+    65us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4844349,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":1399,"apply.run_cpu_time_us":4678,"apply.run_wall_time_us":109607,"num_ops":8,"prepare.queue_time_us":232905,"prepare.run_cpu_time_us":14110,"prepare.run_wall_time_us":92678,"raft.queue_time_us":92946,"raft.run_cpu_time_us":1657,"raft.run_wall_time_us":1656,"replication_time_us":150002,"thread_start_us":295981,"threads_started":4,"wal-append.queue_time_us":42746,"wal-append.run_cpu_time_us":25447,"wal-append.run_wall_time_us":1207866}]]}
I20260709 12:10:48.834623 13330 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.857196 13283 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.846410 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.862030 10580 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:48.863312 13122 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.863801 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:48.864972 13071 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:48.865667 13071 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.866751 13330 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.868165 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.874387 13071 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.839677 13139 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.878312 11218 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:48.879069 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:48.870072 13345 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.884009 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.884402 13345 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.869848 13122 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.871999 13243 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:48.886909 13330 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:48.877897 13333 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:48.896673 13330 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:48.899772 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 228524us
I20260709 12:10:48.902251  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:48.903187 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.903353 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.904784 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.892092 13285 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:48.915803 13255 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:48.919785 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.920842 13207 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:48.921427 13333 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.893908 13243 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:48.937008 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.937214 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:48.939285 13206 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:48.906733 13330 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:48.945785 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:48.924538 13122 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:48.947063 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:48.939903 13206 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:48.939888 13212 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.941584 13243 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.943858 13295 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:48.968552 13139 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:48.968777 13290 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:48.969478 13290 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:48.969822 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:48.970850 13212 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:48.970585 13290 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.971397 13212 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:48.971315 13206 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.972046 13212 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:48.961414 10322 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:10:48.975343 10322 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 69608.4us
I20260709 12:10:48.936223 13255 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.977927 13255 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.903072 13283 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:10:48.983827 13283 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:48.984237 13283 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:48.985267 13283 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:48.948921 13258 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:48.979564 13368 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.139s	user 0.000s	sys 0.002s
I20260709 12:10:48.946239 13176 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:48.987915 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:48.966351 13155 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:48.998898 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:48.959117 13305 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:48.987190 13307 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:48.974743 12945 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.013574 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:49.018813 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:49.025777 12945 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.032050 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:49.032894 11219 raft_consensus.cc:2376] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8. Re-sending same reply.
I20260709 12:10:49.039677 11220 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:49.040084 13333 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.001804 13139 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:49.002306 13258 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:49.014686 13283 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.012818 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.047533 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:49.059523 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.061718 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.063753 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.060323 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.069607 13333 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:49.070369 13333 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.071220 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.071789 11218 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:49.071902 11212 raft_consensus.cc:2376] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7. Re-sending same reply.
I20260709 12:10:49.072877 13333 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.022730 13345 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:10:49.073868 13345 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:49.073935 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.074329 13345 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:49.075528 13345 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.075590 13258 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.069655 13307 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.076872 13345 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.078342 13307 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.058277 12945 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:49.079037 13236 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.079630 13236 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.081727 13236 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.082310 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.043642 13139 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:49.117973 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 171857us
W20260709 12:10:49.118784 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.107582 11213 raft_consensus.cc:2393] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:49.128129 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 119929us
I20260709 12:10:49.087379 13139 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:49.131719 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.134176 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.137853 11220 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:49.086751 11217 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:49.091456 13122 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:49.152750 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:49.154394 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 214303us
I20260709 12:10:49.151734 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:49.134817 13243 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:10:49.171489 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 124062us
I20260709 12:10:49.174086 11216 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.179234 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:49.182242 13212 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.184724 13212 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:49.189594 13387 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:49.190403 13387 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:49.181567 13251 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:49.193121 10403 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:10:49.195923 10403 leader_election.cc:403] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:10:49.201614 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:49.203943 13386 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:10:49.204614 13387 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.205801 13382 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:49.213795 13122 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:49.220999 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47518.3us
I20260709 12:10:49.224102 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:49.235697 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 64376.1us
I20260709 12:10:49.239308 13251 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:49.251116 13307 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
W20260709 12:10:49.265549 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.263638 12659 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:49.272660 13330 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.273831 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:49.277803 13376 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 12:10:49.279201 13382 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:10:49.280159 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 78646.7us
I20260709 12:10:49.281533 13382 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:49.281985 13382 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.282362 13382 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:49.282737 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.283449 10511 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:49.284147 12659 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:49.283200 13390 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:49.284961 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.285687 10506 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:49.286828 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.283366 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.287506 10509 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:49.287719 10434 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:49.288611 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.289250 10505 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:49.290596 10508 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:49.285079 13390 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:49.291453 12659 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.291735 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.293679 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.294449 10512 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:49.291846 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 56603.8us
I20260709 12:10:49.304940 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:49.305651 10585 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:49.306854 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:49.307523 10585 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:49.274381 13139 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.316105 11216 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:49.317116 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:49.320013 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.328104 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.328279 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.329102 10430 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:49.330224 10429 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:49.291859 13390 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:49.331337 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.319265 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:49.321702 10580 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:49.322268 10504 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:49.330228 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.339401 13390 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.339874 11213 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:49.340045 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.336504 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:49.341027 10587 raft_consensus.cc:2368] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8.
I20260709 12:10:49.341214 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:49.339191 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 43770.3us
I20260709 12:10:49.344493 13387 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:49.346647 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.347333 10433 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:49.349321 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 3.724s (SENT) [suppressed 21 similar messages]
I20260709 12:10:49.357357 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.358314 10436 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:49.359794 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.360946 10435 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:49.362534 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:49.364972 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53344.5us
I20260709 12:10:49.366832 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.367501 10512 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:49.370755 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.375738 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:49.376791 10513 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:49.310904 13251 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:49.382265 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:49.330909 13212 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:49.397485 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:49.397082 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47781.8us
I20260709 12:10:49.398566 10512 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:49.399534 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.334846 13345 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:10:49.401626 11221 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:49.400063 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:49.404377 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.401541 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.378731 13258 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.403589 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.424501 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:49.408445 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.425367 10513 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.425546 10432 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:49.408533 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.427141 10429 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:49.412004 10436 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:49.412789 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.429334 10430 raft_consensus.cc:2368] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7.
I20260709 12:10:49.414702 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:49.430047 10504 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.418931 10510 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:49.420287 10580 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:49.441319 10434 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:49.442822 10434 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:10:49.443125 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.443688 13243 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.443819 11216 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:49.407171 13374 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:10:49.445511 13374 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:49.450084 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.450891 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.451021 10512 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:10:49.454327 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54382.6us
I20260709 12:10:49.455852 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:49.459072 10506 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:49.468165 10436 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:49.400449 13212 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.438593 13305 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:49.423357 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.473579 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.475924 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.478063 10435 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:49.457746 13374 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:49.502640 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.504354 10506 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:49.508417 10434 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:49.514338 10513 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:49.520316 13139 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:49.437358 13258 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.523818 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.536329 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.499671 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.530139 10429 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:10:49.508498 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 52772.2us
I20260709 12:10:49.474527 13243 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:49.522083 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:49.536116 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.531579 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:49.522154 13212 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.556581 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.557792 10430 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:49.558403 13212 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:49.559046 13212 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.559514 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:49.560691 13258 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.561256 13212 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:49.551981 13390 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:49.561762 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:49.562384 13390 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:49.562821 13390 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:49.563391 11216 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:49.563498 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.564194 10429 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:49.562799 13212 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:49.552260 10438 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:49.567951 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.569125 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.569664 10438 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.561787 13258 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.572484 13258 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.572851 13258 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.558799 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.575412 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.576264 10435 raft_consensus.cc:2368] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7.
I20260709 12:10:49.587792 13374 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.588987 13374 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:49.591303 13381 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:49.595260 10429 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:49.550551 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:49.446909 13382 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.506625 13372 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:10:49.555291 13305 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:49.556411 13387 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:49.549367 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.461668 13251 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:49.621387 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.522356 13330 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.509687 11213 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:49.628345 10510 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:49.628152 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.633958 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.641608 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.586714 13390 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:49.732457 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104367us
I20260709 12:10:49.655910 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:49.655956 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 105517us
I20260709 12:10:49.734050 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.735613 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:49.693255 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.708189 10438 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:10:49.710142 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:49.711875 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53632.7us
I20260709 12:10:49.653486 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:49.750804 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.752586 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.736743 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.714596 13214 raft_consensus.cc:2764] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:10:49.759821 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.749269 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:49.763968 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.750399 13330 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:49.767186 10429 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:49.800885 10433 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:49.767297 13307 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.767865 13393 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:49.769002 13402 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:10:49.769536 10430 raft_consensus.cc:2368] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:10:49.775070 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.776968 13301 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.778784 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:49.844347 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66021.3us
I20260709 12:10:49.783298 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:49.783957 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:49.796736 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.805354 13212 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:49.803369 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.722425 13387 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:49.807189 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44343.2us
I20260709 12:10:49.817631 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.854743 10434 raft_consensus.cc:2410] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:10:49.818851 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.855340 10436 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.855792 10433 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:49.820339 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.857117 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.858036 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:49.823251 10504 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:49.824949 11216 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:10:49.860224 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:49.826236 10432 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:49.839689 10429 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:49.801043 13239 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:49.804777 13243 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:49.870324 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.802559 13330 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.874301 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.800748 13379 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:10:49.851893 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:49.884536 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.885177 10434 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.727274 13305 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.848869 10510 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:49.845355 13301 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:49.813987 13307 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:49.829344 13393 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.878746 10504 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:10:49.854280 13212 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.895439 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.895583 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.896436 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:49.910683 10430 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:49.870682 13243 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.898473 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.868476 13239 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.898006 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:49.869551 13236 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.869697 13192 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.856659 12659 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.855295 13387 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:49.882025 13330 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.936548 10507 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:49.938747 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:49.944078 11216 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:49.938907 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:49.940120 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.941097 10504 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:49.940966 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.893543 13283 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:49.899207 13301 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.862826 13391 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:10:49.898159 13214 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.950974 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.862283 13122 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:10:49.897518 13382 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:49.963881 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.922266 13212 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:49.926205 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:49.975026 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 77106us
I20260709 12:10:49.940172 13192 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.909943 13393 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.940325 13239 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.945781 13206 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:49.982254 10430 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:49.939896 13236 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.939778 12659 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.988572 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.947412 13243 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:49.939910 13335 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:50.003312 13236 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.003408 12659 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:50.015218 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.990682 10504 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:50.016307 13305 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:49.852545 13381 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:49.992246 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.000838 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:49.968992 13214 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.004384 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:50.001781 10429 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:50.033922 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.039063 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:50.035295 10435 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:49.987373 13192 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.981350 13283 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:50.040092 10438 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:50.041278 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.046532 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.046787 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.047446 10430 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:50.046821 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.000355 13206 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.051178 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.054491 10434 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:50.037930 11213 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:50.055382 10430 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:50.055032 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:50.031910 13214 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.053804 13206 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.041838 13345 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.066198 10435 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:10:50.085261 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.079439 10429 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:50.080171 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.070825 10433 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:50.042409 13372 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:10:50.090610 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.092949 10433 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:50.098815 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.100045 10438 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:50.048720 13258 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.105521 10434 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:50.133592 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.146770 10432 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.103415 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.138409 10584 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:10:50.148011 10430 raft_consensus.cc:2368] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:10:50.105614 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.148986 10435 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:10:50.139894 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 249237us
I20260709 12:10:50.157866 10433 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:50.165560 10478 leader_election.cc:400] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8.
I20260709 12:10:50.165987 10478 leader_election.cc:403] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:10:50.166735 10434 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:50.102504 13374 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:50.177244 10435 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:50.171816 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.102025 13390 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:50.151325 13411 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:10:50.189815 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:50.190843 10433 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:10:50.191308 10433 pending_rounds.cc:85] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:10:50.191977 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.192602 10430 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:50.201817 10322 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:50.157382 13283 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:50.191531 10433 pending_rounds.cc:107] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:10:50.205914 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:50.206393 10433 raft_consensus.cc:2889] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:50.206501 10436 raft_consensus.cc:2410] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:50.206602 10507 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.210906 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.211635 10431 raft_consensus.cc:2376] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6. Re-sending same reply.
I20260709 12:10:50.211757 13176 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:10:50.215380 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.217062 10507 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:50.223428 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:50.192474 13374 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:50.252779 11215 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:10:50.222770 13382 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:50.240574 13390 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:10:50.276062 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.276889 10431 raft_consensus.cc:2376] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8. Re-sending same reply.
I20260709 12:10:50.278935 11213 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:50.234829 13379 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:10:50.229980 12659 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.240784 13391 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:10:50.274550 10432 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:50.270237 13307 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.268886 13425 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.305934 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.306349 13374 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:50.270682 13382 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:50.312249 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.350437 10430 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
W20260709 12:10:50.366460 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 176778us
I20260709 12:10:50.284605 13258 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:50.371366 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 148518us
I20260709 12:10:50.375453 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:50.352533 13305 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8.
W20260709 12:10:50.381960 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.406067 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.376127 13422 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:50.411146 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.424417 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.370959 13425 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.524271 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.525106 10438 raft_consensus.cc:2376] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7. Re-sending same reply.
I20260709 12:10:50.525846 13425 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:50.440244 13426 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:10:50.440476 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.526298 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.527253 10437 raft_consensus.cc:2410] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:10:50.441187 13427 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:10:50.441946 13417 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 12:10:50.463099 13420 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:10:50.531582 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.474237 13431 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.532877 13431 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.515730 13382 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:50.480840 12659 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:50.486074 13236 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:50.535835 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.537274 13236 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.467491 13330 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:50.538071 13212 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.000s
I20260709 12:10:50.489534 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.540020 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.540452 10432 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.543339 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.544708 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.545888 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.546497 10430 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
W20260709 12:10:50.547187 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.547757 13431 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.540052 13330 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:50.512005 13301 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.526s	user 0.007s	sys 0.004s
I20260709 12:10:50.513685 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.549733 13214 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:50.550297 10434 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:50.551308 10583 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:50.514254 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:50.551551 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 184210us
I20260709 12:10:50.479918 13192 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.515172 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.517215 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:50.523679 13422 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:50.522918 10431 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7. Re-sending same reply.
I20260709 12:10:50.528463 10438 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:50.527335 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.486491 13393 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.530562 13425 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:50.463696 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:50.564582 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 191948us
W20260709 12:10:50.526319 13239 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.003s
I20260709 12:10:50.567773 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:50.538579 13212 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.558554 10432 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:50.552294 10433 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:50.561272 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.569743 10437 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:50.574237 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:50.575243 10430 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:50.575913 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:50.577518 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:50.583173 10513 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:50.564934 13425 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.561295 13422 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.564851 13330 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.592989 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.572149 12659 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:50.596386 10429 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:50.598066 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:50.597502 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:50.598960 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:50.572134 13236 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.590605 13425 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.601125 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:50.607461 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:50.587201 13139 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.590792 13258 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.609464 10510 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:50.588995 13423 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:50.622892 13423 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:50.610162 13236 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.623435 13423 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.600087 13439 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:50.624531 13408 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.641s	user 0.009s	sys 0.002s
W20260709 12:10:50.624929 13243 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.584s	user 0.014s	sys 0.000s
I20260709 12:10:50.625497 13236 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.625365 13243 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:50.626559 13387 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.673s	user 0.003s	sys 0.010s
I20260709 12:10:50.626972 13387 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.595281 13206 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.629189 13387 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:50.630904 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.619032 10578 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.612514 10430 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:50.634408 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:50.610018 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:50.634743 13206 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.635074 10580 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.616550 10582 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:50.636490 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:50.637060 10582 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.619928 10432 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7. Re-sending same reply.
I20260709 12:10:50.621143 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.562212 13301 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:50.573691 13239 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:50.630786 13139 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.640820 13301 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:50.621197 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.642282 13139 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.611028 12659 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:50.642546 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.643409 10585 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:50.643744 12659 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.634538 10430 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:50.645462 10430 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:50.644244 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.648072 10508 raft_consensus.cc:2368] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:10:50.648754 10585 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:50.650202 10586 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:50.611496 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.652258 10581 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:50.663687 10580 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
W20260709 12:10:50.667899 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:50.704177 10553 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.501s (SENT)
I20260709 12:10:50.705022 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:50.727908 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:50.727706 10430 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.603873 13435 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.734956 10582 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:50.738301 10579 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:50.742865 10586 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:50.599699 13330 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:50.750272 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47068us
I20260709 12:10:50.604223 13440 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.144s
W20260709 12:10:50.757452 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 105093us
W20260709 12:10:50.759151 10553 leader_election.cc:336] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.587s (SENT)
I20260709 12:10:50.747128 13122 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:50.767455 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:50.769953 10430 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:10:50.772825 13448 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:10:50.774132 13214 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:50.624611 13439 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:50.779721 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:50.755440 13387 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:10:50.744635 13435 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.073139 10578 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:51.091573 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 305211us
W20260709 12:10:51.008322 10336 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 107) took 12395 ms (client timeout 15000 ms). Trace:
I20260709 12:10:50.880803 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.805891 10513 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:51.098448 10510 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:50.886005 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.103058 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:51.122622 10513 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.831750 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:50.869570  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:51.123831 10505 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:50.776361 13243 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:50.800545 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:51.129318 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.891295 11220 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:10:50.807699 13451 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:10:50.999230 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:50.967947 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 134624us
W20260709 12:10:50.885066 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.133819 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.102766 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.110649 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.107846 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.104100 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:50.902158 13122 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:50.940068 13452 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:10:51.127391 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.128177 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:51.147387 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.147523 10578 raft_consensus.cc:2368] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
I20260709 12:10:51.129264 10507 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:51.099237 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.148960 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.150249 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.912s (SENT)
I20260709 12:10:50.867815 13212 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.135090 10508 raft_consensus.cc:2368] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
I20260709 12:10:50.868075 13192 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.126188 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.118901 13437 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.160216 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:51.100966 10336 rpcz_store.cc:269] 0709 12:10:38.607519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.611209 (+  3690us) service_pool.cc:224] Handling call
0709 12:10:51.002498 (+12391289us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:49.023908 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:49.024696 (+   788us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:49.024760 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:10:49.024778 (+    18us) tablet.cc:662] Decoding operations
0709 12:10:49.026026 (+  1248us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:49.026155 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:49.026185 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:10:49.177170 (+150985us) tablet.cc:701] Row locks acquired
0709 12:10:49.177231 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:10:49.177437 (+   206us) write_op.cc:270] Start()
0709 12:10:49.177603 (+   166us) write_op.cc:276] Timestamp: P: 1783599049177366 usec, L: 0
0709 12:10:49.177644 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:10:49.178442 (+   798us) log.cc:844] Serialized 4666 byte log entry
0709 12:10:49.442094 (+263652us) spinlock_profiling.cc:243] Waited 9.89 ms on lock 0x7b1800090d00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:50.058132 (+616038us) op_driver.cc:464] REPLICATION: finished
0709 12:10:50.431857 (+373725us) write_op.cc:301] APPLY: starting
0709 12:10:50.449209 (+ 17352us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:50.449499 (+   290us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:50.449537 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:50.456109 (+  6572us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:50.456481 (+   372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:10:50.456559 (+    78us) write_op.cc:312] APPLY: finished
0709 12:10:50.457003 (+   444us) log.cc:844] Serialized 83 byte log entry
0709 12:10:50.457207 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:50.457389 (+   182us) write_op.cc:454] Released schema lock
0709 12:10:50.457476 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:50.458006 (+   530us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:50.458050 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":922141,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":347499,"apply.run_cpu_time_us":4492,"apply.run_wall_time_us":35418,"num_ops":11,"prepare.queue_time_us":189250,"prepare.run_cpu_time_us":34234,"prepare.run_wall_time_us":377769,"raft.queue_time_us":612561,"raft.run_cpu_time_us":21030,"raft.run_wall_time_us":243539,"replication_time_us":880300,"spinlock_wait_cycles":22736768,"thread_start_us":1030596,"threads_started":4,"wal-append.queue_time_us":232089}]]}
I20260709 12:10:50.793391 13301 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:50.858206 13283 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.130151 13207 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:10:50.832427 13330 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.147568 10581 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
W20260709 12:10:51.150400 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.098484 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.099390 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 340260us
I20260709 12:10:51.175215 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.025660 13393 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:51.154619 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.038882 13422 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:50.805054 13239 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.161271 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.191145 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.047173 13423 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:51.175836 10583 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:51.175801 10553 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.619s (SENT)
I20260709 12:10:51.199813 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.184150 10579 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:51.184032 10513 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:51.132154 13445 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:10:51.166916 13243 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:51.168946 13390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.024s	sys 0.037s
I20260709 12:10:51.196043 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.126595 13425 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.229905 10509 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:10:51.231772 13425 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.187357 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.232877 10580 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:51.205211 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.211920 10507 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:51.235934 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.213379 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.237445 10582 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:51.237526 10505 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:51.214040 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.238550 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.216737 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:51.219871 10578 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.221072 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:51.241326 10513 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:51.221179 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:51.241688 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 642752us
I20260709 12:10:51.242393 10510 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:51.225045 13423 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
W20260709 12:10:51.225157 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49388.9us
W20260709 12:10:51.243610 10553 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.633s (SENT)
I20260709 12:10:51.244079 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.227739 10552 leader_election.cc:400] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:10:51.244946 10552 leader_election.cc:403] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:10:51.215160 13243 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:51.159250 13122 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.236884 10507 raft_consensus.cc:2410] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:51.238564 10585 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 5: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:51.214926 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64755.4us
W20260709 12:10:51.274609 11178 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.558s (SENT)
I20260709 12:10:51.275370 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.269716 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.277021 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.280122 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.280414 10580 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:51.270031 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:51.213615 13440 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:51.207146 13212 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:51.284556 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.171161 13283 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.185789 13307 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:10:51.299199 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.262581 13258 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:51.240922 13374 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:51.169473 13437 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.272063 13236 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:51.241918 13390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.025s	sys 0.037s
I20260709 12:10:51.287962 13176 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.213212 13251 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.078s
W20260709 12:10:51.302462 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58897.8us
W20260709 12:10:51.331425 10553 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.691s (SENT)
I20260709 12:10:51.296497 13214 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:51.331964 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.268433 13243 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.304070 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.300127 10581 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:51.267346 13425 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:51.315730 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:51.316166 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 375) took 1222 ms. Trace:
I20260709 12:10:51.317145 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.317952 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:51.318418 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 132112us
I20260709 12:10:51.287024 13239 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:51.300711 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.308657 13374 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:51.301223 13330 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:51.287081 13305 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:51.291802 13387 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:51.297187 13423 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:51.337288 10584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55454
I20260709 12:10:51.327430 13230 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:51.393532 10584 raft_consensus.cc:606] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Received request to transfer leadership
I20260709 12:10:51.340849 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.349299 10587 rpcz_store.cc:276] 0709 12:10:50.093915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:50.094065 (+   150us) service_pool.cc:224] Handling call
0709 12:10:50.741262 (+647197us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:10:50.741400 (+   138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:51.129012 (+387612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:51.129042 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:51.129086 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:51.129116 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:51.315911 (+186795us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400256054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:51.315972 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:51.316102 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893687936}
I20260709 12:10:51.302672 13440 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.326325 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.375865 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.389233 10583 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.399533 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.400074 10509 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:51.394627 10580 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:10:51.397782 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 97155.4us
I20260709 12:10:51.409564 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.419330 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.347429 13212 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.420945 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.421413 10510 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:51.422921 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.424750 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.347699 13258 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.430907 10509 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:10:51.434161 10512 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:51.489009 10580 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:10:51.484015 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.484719 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.397631 13433 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:51.393257 13390 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:51.496502 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.412283 13192 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:51.397255 13423 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:51.497949 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.496975 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.397838 13392 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:10:51.516469 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 212477us
W20260709 12:10:51.437088 13465 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:10:51.431242 13416 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.379508 13425 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.409698 13391 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:10:51.333698 13176 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.517632 13307 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.521623 10513 raft_consensus.cc:2393] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:51.409392 13429 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.430130 13330 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.400719 13305 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:51.541971 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.520557 13433 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:51.437707 13387 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:51.548110 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:51.559077 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 227677us
I20260709 12:10:51.527948 13423 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:51.554009 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 146479us
I20260709 12:10:51.556502 10507 raft_consensus.cc:2410] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:10:51.565611 10403 connection.cc:256] RPC call timeout handler was delayed by -2.8066219059999185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
I20260709 12:10:51.412727 13206 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:51.568964 10581 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:51.511834 13439 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.532411 13192 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:51.571426 10403 connection.cc:256] RPC call timeout handler was delayed by -2.028121906000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3098s for a response
I20260709 12:10:51.585574 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:51.572183 10553 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.771s (SENT)
I20260709 12:10:51.575651 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.538358 13435 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:51.588768 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 157830us
I20260709 12:10:51.589912 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:51.591605 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.591856 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.573868 13433 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:51.606223 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.571961 13422 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.607920 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:51.610666 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.537025 13416 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:51.611908 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.569722 13214 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:51.613564 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:51.605517 13453 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260709 12:10:51.570382 13440 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.617012 10586 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
W20260709 12:10:51.616966 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44839.5us
I20260709 12:10:51.616986 13301 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:51.522930 13374 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.624351 10508 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:51.605727 13438 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:51.555732 13307 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.635274 13438 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:51.638118 13438 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.639083 13307 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.639528 13438 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.641670 13416 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.633654 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.642838 13214 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:51.643950 10581 raft_consensus.cc:2368] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
I20260709 12:10:51.625312 13474 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.650167 13474 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.653326 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.654621 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:51.653512 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.655424 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.657382 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.624378 13283 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.660358 10509 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:51.661090 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 127913us
I20260709 12:10:51.661809 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:51.662142 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:51.669536 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.674561 10508 raft_consensus.cc:1217] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[7.5-7.5]   Dedup: 7.5->[]
I20260709 12:10:51.675100 10508 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:10:51.677423 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 87602.2us
I20260709 12:10:51.605355 13212 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.634246 13477 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:51.682266 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53751.1us
I20260709 12:10:51.591607 13258 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.679240 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:51.688926 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:51.660331 13416 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:51.651420 13243 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.692358 10586 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:51.668784 13374 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.695353 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:51.698628 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.838s (SENT)
I20260709 12:10:51.701738 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.706380 10578 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:51.708607 13464 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:10:51.710179 13243 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.703703 13476 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:51.711439 13476 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.713716 13251 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:51.710188 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1612) took 1511 ms. Trace:
I20260709 12:10:51.662925 13330 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.712486 13476 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.714998 10511 rpcz_store.cc:276] 0709 12:10:50.198953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:50.199202 (+   249us) service_pool.cc:224] Handling call
0709 12:10:50.207131 (+  7929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:50.407211 (+200080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:50.407260 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:51.171327 (+764067us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b2c0001c5e8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:10:51.201207 (+ 29880us) log.cc:844] Serialized 26 byte log entry
0709 12:10:51.297268 (+ 96061us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:51.297311 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:51.297395 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:51.709851 (+412456us) raft_consensus.cc:1685] finished
0709 12:10:51.709881 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:51.710117 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16305,"raft.queue_time_us":226616,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":15679,"spinlock_wait_cycles":4074880,"thread_start_us":268881,"threads_started":2,"wal-append.queue_time_us":164838}
I20260709 12:10:51.718053 13437 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.700373 13176 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.682112 13206 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:51.702744 13373 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.724193 10403 leader_election.cc:400] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
W20260709 12:10:51.733963 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.734143 10403 leader_election.cc:403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:10:51.729123 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.700322 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.735680 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.736347 10582 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:51.737061 10553 leader_election.cc:336] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.839s (SENT)
I20260709 12:10:51.737926 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.699996 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:51.741176 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:51.744323 10509 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:51.746269 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:51.746892 10579 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.746891 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.751924 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.753001 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.753616 10430 raft_consensus.cc:2376] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8. Re-sending same reply.
I20260709 12:10:51.754333 10512 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:51.755385 10432 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:10:51.664757 13387 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.760121 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.761591 10510 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:51.762027 10510 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.696089 13416 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.767992 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.723748 13474 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.772150 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:51.762928 10510 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.777673 10510 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:51.770879 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.774449 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:51.747152 13243 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:51.784502 13243 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:51.786654 13243 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.776312 13381 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:51.786944 13466 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:51.787462 13381 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:51.788151 13435 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:10:51.788421 13381 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.788909 13484 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:10:51.789608 13381 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:51.677577 13192 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.748499 13425 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:51.751328 13483 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:51.748674 13206 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:51.795802 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.797441 10508 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:51.797955 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.798900 10433 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:51.801095 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.801833 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.804412 10433 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.804715 10435 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.807463 10508 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:51.808172 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.802847 10434 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:51.808986 10433 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.809104 10508 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:51.810649 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.811409 10436 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:51.782541 13283 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.819052 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.819788 10430 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:51.785233 13258 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:51.842130 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.845228 10431 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.845678 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.825095 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.846369 10430 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:51.791744 13473 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.791539 13441 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:51.842635 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:51.791134 13484 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:10:51.848372 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:51.852454 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:51.792851 13474 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:51.854626 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:51.855722 11216 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.868126 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:51.859673 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.869354 10438 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.870522 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.871143 10438 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.742719 13373 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.873977 10579 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.871446 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:51.855155 13441 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:51.878067 11221 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:51.825860 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.822793 13425 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:51.828539 13440 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:51.844317 13258 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:51.798934 13466 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:10:51.886092 10433 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.886087 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:51.889451 10430 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:51.863549 13437 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:51.898399 13437 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:51.890667 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.899675 10437 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:51.900745 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.901391 10437 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:10:51.902581 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:51.891013 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.003s (SENT)
I20260709 12:10:51.903265 10430 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.891582 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.897318 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:51.897645 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:51.905891 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 111212us
W20260709 12:10:51.897900 13490 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:10:51.898875 13437 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.906574 10433 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.916239 10403 leader_election.cc:400] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
I20260709 12:10:51.879896 13441 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.867429 13477 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.849731 13283 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:51.927389 10431 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.928089 10433 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.913650 10438 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.929350 10430 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.929316 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.854420 13330 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:10:51.935837 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:51.935628 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:51.917886 10403 leader_election.cc:400] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:10:51.938591 10431 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.930490 10438 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:51.938833 10437 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.939038 10513 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:10:51.938704 10403 leader_election.cc:403] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:51.942009 11216 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.895771 13258 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.894418 13251 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:51.875068 13474 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.929934 13483 raft_consensus.cc:2764] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:10:51.955679 10585 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:51.957535 13488 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:10:51.958439 13283 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.903203 13307 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:10:51.961226 13307 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:51.961649 13307 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:51.890452 13425 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:51.943080 13478 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.932649 13230 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:10:51.970709 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:51.976114 11220 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
W20260709 12:10:51.970427 13482 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:10:51.992262 10431 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:51.968541 13283 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:10:51.964955 13484 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:51.999104 13307 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.002424 13425 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.005278 13494 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:10:52.005757 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:52.006482 13416 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:51.976567 13483 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:52.017179 13243 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.028259 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:52.022763 10435 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
W20260709 12:10:52.052836 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 86349.5us
I20260709 12:10:52.053359 10431 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:51.936594 13477 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:52.051760 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 97608.7us
I20260709 12:10:52.055922 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:51.943351 13206 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:52.065374 10430 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:52.059577 13307 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:52.036248 13484 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.053575 13230 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:51.977236 13478 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.062326 13416 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.086078 13243 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.071904 13230 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:52.092312 10510 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:10:52.096215 10510 pending_rounds.cc:85] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 7
W20260709 12:10:52.084643 13492 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:10:52.101254 10510 pending_rounds.cc:107] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:10:52.101578 10510 raft_consensus.cc:2889] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:52.103659 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:52.105325 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:52.106846 13495 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:52.107607 10429 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:52.106632 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:52.109127 10431 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:52.109663 13477 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.111248 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:52.111528 13484 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.111920 11213 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.116129 13478 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.121534 13437 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.124158 13122 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.125902 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.127105 10512 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 6: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:52.128350 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:52.160419 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.131489 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.161592 10510 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:10:52.165658 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:52.130854 13429 consensus_meta.cc:287] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.003s
I20260709 12:10:52.173120 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:52.169973 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:52.103149 13243 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:52.179193 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:52.103444 13373 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.183326 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1619) took 1598 ms. Trace:
I20260709 12:10:52.164228 13416 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:52.182233 13374 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:52.228075 13374 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:52.228658 13374 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.187849 13258 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:52.199827 13236 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.878s	user 0.016s	sys 0.000s
I20260709 12:10:52.235051 13236 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:52.235257 10505 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:52.205466 13416 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.179816 13429 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.205896 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:52.103789 13283 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.102s
I20260709 12:10:52.209837 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:52.212440 13499 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:10:52.227197 13487 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:10:52.184579 10506 rpcz_store.cc:276] 0709 12:10:50.584637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:50.585784 (+  1147us) service_pool.cc:224] Handling call
0709 12:10:50.600659 (+ 14875us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:51.237220 (+636561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:51.237266 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:51.289420 (+ 52154us) log.cc:844] Serialized 26 byte log entry
0709 12:10:51.486064 (+196644us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:51.486107 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:51.486181 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:52.118739 (+632558us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b2400167438. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa30fb0 00007f2c926a7488 00007f2c926a7be8 00007f2c9267f398 00007f2c9267a76a 00007f2c92679bb6
0709 12:10:52.172356 (+ 53617us) raft_consensus.cc:1685] finished
0709 12:10:52.172387 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:52.183202 (+ 10815us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":461419,"raft.queue_time_us":484845,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":1774,"spinlock_wait_cycles":38335232,"thread_start_us":353319,"threads_started":2,"wal-append.queue_time_us":288476}
I20260709 12:10:52.183717 13474 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.103761 13483 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:10:52.243446 10437 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:52.239254 13433 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.615s	user 0.015s	sys 0.000s
I20260709 12:10:52.249961 13416 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.241161 13258 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.244987 13230 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.246537 13373 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:52.253361 13474 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.187798 13243 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.254139 13473 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
I20260709 12:10:52.254478 10583 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:52.272545 13373 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:52.261209 13423 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.006s
I20260709 12:10:52.279464 13423 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.282529 13423 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.260496 13307 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:52.242846 13438 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.004s
I20260709 12:10:52.254235 11213 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:52.283779 10429 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:52.296228 10403 leader_election.cc:400] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
W20260709 12:10:52.298408 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.309772 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.310217 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:52.310361 13498 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:52.310488 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:52.311484 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.313046 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.314024 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.315241 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.320851 11221 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:52.322340 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:52.333376 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:52.334120 10429 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:52.301754 13473 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:52.340864 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:52.269951 13433 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.307591 13230 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:52.384541 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:52.386855 13373 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.245714 13441 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.307957 13437 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.332546 13423 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:10:52.349859 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.372747 10436 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:52.404042 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.374809 13493 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:52.376237 10335 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 3 to 4, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 4 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:52.351742 13416 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:52.381819 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.381944 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 644885us
I20260709 12:10:52.405212 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.406962 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.382272 13243 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.382371 13481 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.370452 10437 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:52.416060 10504 raft_consensus.cc:2368] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:10:52.390056 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:52.391889 10433 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:52.393380 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:52.339709 13473 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:52.340049 13483 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.283493 13474 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.403832 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.308035 13307 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.359470 13192 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:52.418535 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:52.421885 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.378013 13283 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:52.412185 13416 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:52.446352 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.441568 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.453588 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63619us
I20260709 12:10:52.401197 13440 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:10:52.420506 13423 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:52.494112 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 179129us
I20260709 12:10:52.453394 13483 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.499747 10429 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:52.427390 13445 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:52.424117 13511 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.496876 10479 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:52.428681 13493 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:52.498551 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.494154 13192 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:52.494845 13459 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:52.501760 13445 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:52.512758 13460 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:52.509302 13381 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.719s	user 0.018s	sys 0.003s
I20260709 12:10:52.520532 13423 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.522495 13433 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.523552 13441 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.523825 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2037) took 1321 ms. Trace:
I20260709 12:10:52.546312 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.546554 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.547094 10579 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:52.546587 10504 rpcz_store.cc:276] 0709 12:10:51.202417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.203252 (+   835us) service_pool.cc:224] Handling call
0709 12:10:52.523549 (+1320297us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540036ea54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:52.523733 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704000896}
W20260709 12:10:52.526866 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 633892us
I20260709 12:10:52.527695 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:52.552913 13470 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.950s	user 0.004s	sys 0.000s
I20260709 12:10:52.536165 13214 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:52.497663 13498 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:52.566669 10578 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:52.567160 13520 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:10:52.568039 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:52.568518 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.568109 10335 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 2, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:52.569483 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:52.570554 13441 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:52.570676 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.571213 13441 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:52.572275 13441 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.574934 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.575476 13192 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.541849 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.568048 10582 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:52.581139 10335 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 4 to 5, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 5 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:52.543190 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.545126 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.556252 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:52.583982 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.585588 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:52.561584 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.560609 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.588714 10583 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:52.503641 13483 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.563424 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.539029 13460 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:52.594022 13460 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.597527 13305 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.599252 13440 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:52.599956 13440 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:52.600471 13440 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:52.601575 13509 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.410s	user 0.001s	sys 0.001s
I20260709 12:10:52.590332 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.539736 13445 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.603371 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2047) took 1228 ms. Trace:
I20260709 12:10:52.603745 10507 rpcz_store.cc:276] 0709 12:10:51.375364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.375518 (+   154us) service_pool.cc:224] Handling call
0709 12:10:52.603156 (+1227638us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540036d154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:52.603304 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334974208}
I20260709 12:10:52.605568 13307 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:52.607622 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 109154us
I20260709 12:10:52.608563 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.609941 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.585148 13214 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.590852 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.619098 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.591050 10584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:55454
I20260709 12:10:52.589471 13433 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:52.590034 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.592284 13474 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.592944 13483 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:52.620739 10584 raft_consensus.cc:606] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Received request to transfer leadership
I20260709 12:10:52.591779 13459 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:52.625738 10584 raft_consensus.cc:614] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:52.496206 13374 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.627996 13523 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:52.628955 13214 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:52.624933 13517 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.638942 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:52.630105 13506 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:52.606863 13473 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:52.667670 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48690.1us
I20260709 12:10:52.639755 10583 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:52.668716 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.668172 10581 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:52.641382 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:52.642628 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 237072us
W20260709 12:10:52.671840 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.644784 10580 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:52.646103 10585 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:52.674917 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:52.676084 10579 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:52.682505 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:52.683586 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:52.685231 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.656002 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:52.625177 13518 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:52.631889 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:52.689100 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:52.699576 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.807423 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.686137 13523 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:52.717489 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.718372 10579 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:52.686511 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.694922 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:52.865381 10584 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:52.689745 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:52.721199 10553 leader_election.cc:336] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.926s (SENT)
I20260709 12:10:52.716763 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:52.668403 13433 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:52.669176 13495 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:52.856424 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 224621us
I20260709 12:10:52.710553 13473 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:52.859860 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.701891 13517 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.866843 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:52.926405 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 205229us
W20260709 12:10:52.928290 10553 leader_election.cc:336] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.039s (SENT)
I20260709 12:10:52.928855 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:52.869292 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 180312us
I20260709 12:10:52.931393 13438 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.931895 10403 leader_election.cc:400] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
I20260709 12:10:52.932448 10403 leader_election.cc:403] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:52.885775 10583 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:52.889768 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:52.937323 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47654.7us
W20260709 12:10:52.937777 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 238654us
I20260709 12:10:52.895799 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:52.893078 13192 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:10:52.893816 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:52.894840 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:52.894932 13512 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.402s	user 0.002s	sys 0.000s
I20260709 12:10:52.710337 13305 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.167s
I20260709 12:10:52.709566 13214 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.168s
I20260709 12:10:52.702297 13437 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.164s
I20260709 12:10:52.870055 13506 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:52.867909 11220 raft_consensus.cc:2368] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6.
I20260709 12:10:52.934458 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:52.918637 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:52.907915 13433 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:52.958695 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.975183 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.889573 13483 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.958196 13459 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:52.952682 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:52.940852 10553 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.197s (SENT)
I20260709 12:10:52.983487 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:52.896451 13516 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 12:10:52.979259 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.979816 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.988981 13495 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:52.989949 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49119.2us
I20260709 12:10:52.990321 13438 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.960305 13230 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:52.951792 13493 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7.
W20260709 12:10:52.993304 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 337459us
I20260709 12:10:52.970602 13506 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.019793 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:52.996609 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.996977 13533 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:52.996865 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.027406 10507 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:52.997378 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:52.994695 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.999183 13495 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:52.997695 13437 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:53.002074 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:53.030745 10587 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:53.002235 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:52.975662 13523 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:53.003197 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:53.003849 11217 raft_consensus.cc:2410] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:10:53.003664 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.040174 10430 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:53.004431 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.041375 10429 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:53.005002 13236 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.005228 13305 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.005920 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.006641 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.007395 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.048653 10435 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:53.008402 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.049877 10433 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:53.009262 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.011204 13471 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.012790 13477 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:53.054090 13477 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:53.013968 13460 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:52.962414 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:53.055614 11213 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:53.027276 10584 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:53.001353 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:52.992246 13243 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.032131 10436 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:53.026300 13506 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.052563 13437 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:53.057174 13460 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.003713 13532 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:10:53.060662 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51522.8us
I20260709 12:10:53.063401 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.046203 13374 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.039198 13459 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.057914 13393 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.061595 13437 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:53.065526 13493 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.071342 13523 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.049393 13445 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.073555 13532 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:53.064337 13305 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.059875 13236 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:52.979243 13518 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:10:53.053921 13535 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:10:53.096601 10432 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:53.088114 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.084497 13437 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.118583 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:53.129935 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 105099us
I20260709 12:10:53.122937 13483 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.125335 13533 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:53.138428 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:53.133657 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.124727 13523 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:53.127728 13495 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:53.141564 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:53.142293 10583 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.133224 13518 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:10:53.144558 13518 consensus_queue.cc:1243] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:53.145120 13518 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:53.146808 13445 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.134447 13474 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:10:53.147270 13539 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:53.147408 13474 consensus_queue.cc:1243] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:10:53.148445 13495 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.148916 13495 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.149008 13431 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:53.147492 13445 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:53.134502 13440 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.006s
I20260709 12:10:53.151232 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:53.149637 13431 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:53.157536 10583 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:53.171015 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.172744 10504 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:53.172591 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.172933 10507 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:10:53.173641 13483 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.174118 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.174926 13483 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:53.175356 10508 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:53.176147 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.176916 13483 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:53.177022 10508 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:53.177893 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.178172 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.177328 10507 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:53.179672 10403 leader_election.cc:400] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:10:53.180050 10403 leader_election.cc:403] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:53.180644 10510 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:53.150060 13523 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:53.183723 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:53.195351 10510 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:53.181442 13534 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:53.183902 13483 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:53.200387 13542 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:10:53.148041 13539 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.204176 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.193696 13493 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.207290 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.217288 10434 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
W20260709 12:10:53.207957 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:53.200507 13536 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:10:53.219676 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:53.220413 10505 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.222178 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:53.222831 11216 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.169723 13307 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:10:53.212356 10432 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:53.211336 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.053254 13471 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.167485 13445 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.157713 13537 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:10:53.215358 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:53.200587 13538 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:10:53.218554 10429 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:53.193504 13544 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:53.215485 13214 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.223073 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:53.223568 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:53.213939 13530 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:10:53.226039 10510 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:53.170862 13440 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.210258 13534 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:53.231882 13307 consensus_queue.cc:1243] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:53.251345 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.204420 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.214143 13476 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:53.233525 10431 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.208858 10430 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:53.243803 10434 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.159369 13532 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:53.254177 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49857us
I20260709 12:10:53.259053 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:53.259914 13474 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.260355 13474 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.260732 13474 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.272373 10434 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:53.277515 10505 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:53.280395 10437 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:53.255424 13534 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.257020 13518 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.305703 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:53.255159 13460 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.318509 10431 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:53.278637 13305 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.287942 11216 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:53.264858 11213 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: Advancing to term 8
I20260709 12:10:53.242383 13214 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.326262 13214 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:53.326329 13392 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:53.327760 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 261) took 1075 ms. Trace:
I20260709 12:10:53.328197 10509 rpcz_store.cc:276] 0709 12:10:52.252833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:52.255050 (+  2217us) service_pool.cc:224] Handling call
0709 12:10:52.610390 (+355340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:52.610643 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:52.611029 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 12:10:52.982003 (+370974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:52.982064 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:52.982122 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:52.982202 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:53.327561 (+345359us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540036d150. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:53.327687 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":183742,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":902084224,"thread_start_us":370517,"threads_started":2,"wal-append.queue_time_us":305573}
I20260709 12:10:53.331166 13507 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.331831 13507 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.333892 11213 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:53.334331 13507 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.334434 13416 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.335016 13480 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:10:53.240236 13523 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.336301 13480 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:53.336344 13416 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:53.336810 13480 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:53.337721 13539 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.337985 13480 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:53.339790 13541 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:10:53.285008 13387 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:10:53.343026 13387 consensus_queue.cc:1243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:10:53.343609 13387 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:53.343976 13387 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:53.344908 13387 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.346848 10507 pending_rounds.cc:85] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 6
I20260709 12:10:53.348289 10507 pending_rounds.cc:107] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:10:53.348593 10507 raft_consensus.cc:2889] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:53.243216 13431 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.283480 13476 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:53.351637 10322 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:53.346428 13387 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:53.355980 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50438.2us
W20260709 12:10:53.366133 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116565us
I20260709 12:10:53.365164 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.218154 13533 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:53.244417 13544 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.336858 13523 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.349107 13440 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.362546 13483 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.321370 13460 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:53.373718 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89891us
I20260709 12:10:53.374097 13330 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:10:53.382493 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.375514 13441 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:53.365667 13122 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:53.379427 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.383390 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.391613 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:53.346899 13456 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:10:53.394884 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:53.381253 13460 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:53.400180 13545 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.154s	user 0.002s	sys 0.000s
I20260709 12:10:53.384526 13330 consensus_queue.cc:1243] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:53.392086 13498 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.408581 13548 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:53.395198 13122 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.413196 13122 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:53.413471 13243 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:53.414024 13243 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.402036 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:53.416371 11213 raft_consensus.cc:2410] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:10:53.418582 13445 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:53.419454 10384 auto_rebalancer.cc:893] Removing replace marker failed: Replica 33d77e3063684586a737bccf358786d8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:10:53.420047 10384 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:10:53.420360 10384 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:10:53.423386 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:53.424232 11217 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.395028 13416 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.396273 13374 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:53.442660 13416 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.443440 13374 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:53.449187 13544 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.448982 13374 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:53.451191 13473 consensus_meta.cc:287] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.531s	user 0.015s	sys 0.000s
I20260709 12:10:53.403669 13214 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 12:10:53.452957 10553 connection.cc:256] RPC call timeout handler was delayed by -1.753691052000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26320000000000005s for a response
I20260709 12:10:53.453186 13214 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:53.453642 13214 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:53.455844 13471 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.457656 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.458310 10435 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.459146 13556 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:10:53.448936 13493 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
W20260709 12:10:53.453663 10553 connection.cc:256] RPC call timeout handler was delayed by -1.5934364309998728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 12:10:53.467790 10553 connection.cc:256] RPC call timeout handler was delayed by -1.4339175339998746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30210000000000004s for a response
I20260709 12:10:53.403519 13476 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:53.471361 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3306074409999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30560000000000004s for a response
I20260709 12:10:53.462442 13416 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:53.471825 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2224175339999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3256s for a response
I20260709 12:10:53.471880 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:53.472415 10553 connection.cc:256] RPC call timeout handler was delayed by -1.201717533999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3279s for a response
W20260709 12:10:53.472784 10553 connection.cc:256] RPC call timeout handler was delayed by -1.1426292579999426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
I20260709 12:10:53.472743 10429 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:53.475056 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.484047 10580 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:10:53.484932 10580 raft_consensus.cc:606] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:53.485406 10580 raft_consensus.cc:614] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:53.493443 10436 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:53.500147 10435 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:53.507567 10506 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:53.454875 13214 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:53.523808 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 123884us
I20260709 12:10:53.494882 13460 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.526844 10504 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:53.575850 10435 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
W20260709 12:10:53.527402 13559 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 12:10:53.587143 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.589428 13474 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:53.610242 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 85094.1us
I20260709 12:10:53.624969 13493 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.627480 13493 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.578840 13214 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:53.657696 13560 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:10:53.659825 13493 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.661623 13460 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.574620 13437 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.673058 13437 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.673867 10511 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:53.629581 13473 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.675364 13473 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.677552 13473 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.673480 13471 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:53.683440 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:53.684862 11213 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEARNER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:10:53.599191 13495 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.700337 10435 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:10:53.701761 13495 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.703454 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:53.723946 13555 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.323s	user 0.001s	sys 0.002s
I20260709 12:10:53.732091 13474 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.730931 13534 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.791441 10429 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:10:53.792136 10429 pending_rounds.cc:85] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:10:53.792440 10429 pending_rounds.cc:107] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:10:53.792758 10429 raft_consensus.cc:2889] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:53.801802 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 178196us
W20260709 12:10:53.801791 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 189932us
I20260709 12:10:53.795372 13431 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.812681 13387 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:53.816952 13506 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.001s
I20260709 12:10:53.818931 13176 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.814554 13534 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.815374 13556 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:53.825932 13387 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.823156 13456 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:53.829525 13176 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.836938 13387 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:53.837514 13176 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:53.839118 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.840265 10436 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:53.845188 13523 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:53.861649 13523 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.861900 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:53.862906 11217 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:53.864235 13523 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.865571 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:53.854740 13387 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:10:53.854966 13556 consensus_queue.cc:1243] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:53.874718 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:53.875679 13243 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.877784 13243 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.843271 13431 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:53.879155 13572 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:10:53.845029 13567 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:10:53.896003 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.897163 10432 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:53.898072 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:53.898389 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:53.899263 10437 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:53.899480 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:53.894086 13542 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:53.901006 13542 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.903316 13431 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.903518 13176 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:53.905150 13176 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:53.905602 13176 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:53.894732 13573 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.909264 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:53.901022 10437 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:10:53.917075 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.917446 13573 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.919395 13477 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.922827 13542 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.883370 13392 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:53.922413 13176 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.928427 10432 pending_rounds.cc:85] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:10:53.929021 10432 pending_rounds.cc:107] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260709 12:10:53.929597 10432 raft_consensus.cc:2917] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260709 12:10:53.930644 10432 raft_consensus.cc:2889] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:53.902644 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.931998 10431 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.930469 13236 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:53.938304 13573 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:53.939275 13573 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.940503 13122 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:53.940919 13122 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:53.942245 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.943440 10437 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.943722 13176 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.845213 13506 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:53.943877 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:53.981488 10433 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.947646 13416 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:53.950596 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.953800 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:53.955894 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:53.990530 10506 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.990783 10508 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.956297 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:53.957573 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:53.993274 10584 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:10:53.961925 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:53.995117 10578 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:53.995289 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:53.963397 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:53.970919 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:10:53.972508 13524 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.079s	user 0.002s	sys 0.004s
I20260709 12:10:53.953864 13456 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:10:53.975816 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.979264 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:53.997308 10580 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.998288 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:54.024047 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.012389 10431 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:54.036247 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.037065 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.041342 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.047057 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.049515 10505 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:54.060453 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71093.2us
W20260709 12:10:54.061333 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77438.5us
I20260709 12:10:54.070571 11212 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:10:54.075099 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:54.075600 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 77408.7us
I20260709 12:10:54.077096 10581 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.984057 13523 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.006778 10513 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:53.942082 13122 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.071260 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:54.084403 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:54.032095 13456 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:54.089941 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 180782us
I20260709 12:10:54.104791 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.089234 10437 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:54.107867 10433 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:54.080613 13495 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:54.073974 13572 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:10:54.145792 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:54.098565 13416 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:54.152235 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82517.9us
W20260709 12:10:54.153244 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48547.5us
I20260709 12:10:54.138417 13438 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:54.161588 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 263) took 1875 ms. Trace:
I20260709 12:10:54.162132 10438 rpcz_store.cc:276] 0709 12:10:52.286484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:52.377707 (+ 91223us) service_pool.cc:224] Handling call
0709 12:10:52.378115 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:52.458636 (+ 80521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:52.458691 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:52.496751 (+ 38060us) log.cc:844] Serialized 26 byte log entry
0709 12:10:52.636127 (+139376us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:52.667770 (+ 31643us) log.cc:844] Serialized 14 byte log entry
0709 12:10:52.670958 (+  3188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:53.101052 (+430094us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b5400436f54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:53.101109 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:53.932987 (+831878us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:54.161307 (+228320us) raft_consensus.cc:1685] finished
0709 12:10:54.161333 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.161496 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18709,"prepare.queue_time_us":34855,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":48065,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":11097,"spinlock_wait_cycles":264657536,"thread_start_us":138911,"threads_started":1,"wal-append.queue_time_us":378961}
W20260709 12:10:54.166257 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 82141.3us
I20260709 12:10:54.137391 13495 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:54.168226 13495 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.171392 13236 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.172240 13544 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:54.172155 13236 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.172983 13544 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:54.085068 13523 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:10:54.185089 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114142us
I20260709 12:10:54.158648 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.191845 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.181046 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:10:54.195894 13433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.204s	user 0.023s	sys 0.027s
I20260709 12:10:54.197456 10580 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:54.139534 13374 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:54.177409 13431 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:54.253414 13374 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:54.255791 13374 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:54.252635 13433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.261s	user 0.024s	sys 0.027s
I20260709 12:10:54.260540 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.262420 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.175539 13540 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:10:54.222420 13206 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.269495 13206 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.271116 10581 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:54.228034 13429 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:54.276492 13236 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:54.279721 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 123461us
I20260709 12:10:54.280934 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.281837 13564 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:54.296972 10513 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:54.282768 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.298595 10508 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
W20260709 12:10:54.233589 13531 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.316s	user 0.005s	sys 0.012s
I20260709 12:10:54.307241 13429 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:54.287187 13392 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:54.315299 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:54.316792 10506 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:10:54.174976 13544 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.319952 13431 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:54.319156 13239 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:54.286054 13511 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.186136 13416 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.317281 13564 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.203856 13456 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:54.329936 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.302583 13493 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.336122 10438 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:54.334570 10430 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:54.341528 10508 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:54.345453 10513 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:54.331408 13392 raft_consensus.cc:2764] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:10:54.363206 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.366355 13506 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.371404 13572 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.372747 13534 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.373317 13534 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.377432 13416 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:54.378126 13429 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.368844 13564 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:10:54.381181 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.381924 10511 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:54.321000 13431 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.383796 13456 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:54.384039 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1643) took 1209 ms. Trace:
I20260709 12:10:54.384496 10504 rpcz_store.cc:276] 0709 12:10:53.174406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.174555 (+   149us) service_pool.cc:224] Handling call
0709 12:10:53.174836 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:53.528236 (+353400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.383718 (+855482us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540036ef54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:54.383965 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808999168}
I20260709 12:10:54.383706 13534 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:54.389590 13374 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.392066 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.394697 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:54.393735 13206 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.395912 13474 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:54.398226 13564 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.334837 13511 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.341061 13493 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.405489 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.397564 13416 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:54.423065 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 217372us
I20260709 12:10:54.410501 13392 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:54.467645 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.454144 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.454962 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:54.466493 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.470491 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:54.470875 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61397.8us
I20260709 12:10:54.407954 13214 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:54.471935 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:54.478734 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.467608 13381 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.485631 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:54.485994 13587 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:10:54.486837 10582 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:54.486140 13381 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.491416 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:54.501107 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:54.501798 10580 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:54.422909 13523 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:54.497838 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.419250 13416 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.511922 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:54.518471 10581 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:54.512672 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.512781 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.507097 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:54.519984 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:54.496838 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.515179 10513 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:54.515331 10582 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:54.521770 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.523057 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:54.524417 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.526046 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:54.527395 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:54.528337 10579 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:54.528551 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.530043 10585 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:10:54.535547 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.537398 10585 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:54.539170 10585 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:10:54.540293 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.541625 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.549129 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.549311 10580 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:10:54.550840 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.511490 11217 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:54.510547 13392 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:54.552503 10585 pending_rounds.cc:85] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:10:54.554260 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.493307 13586 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:54.554765 10585 pending_rounds.cc:107] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.6
I20260709 12:10:54.506247 13585 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:54.568858 10581 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:10:54.559202 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.509536 13307 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:54.555629 10585 raft_consensus.cc:2917] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Aborting config change with OpId 7.6: Aborted: Op aborted by new leader
I20260709 12:10:54.560997 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.562888 10586 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:54.565788 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:54.558223 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.569948 10432 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:10:54.559809 10434 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:54.624861 10432 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:54.622170 11221 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:10:54.572355 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:54.573264 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:54.626631 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.623505 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.541412 13481 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:54.520985 13506 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.528105 13584 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:54.491272 13214 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.527918 13571 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:54.621038 10438 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:10:54.558820 13392 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:54.642874 10438 pending_rounds.cc:85] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:10:54.643246 10438 pending_rounds.cc:107] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:10:54.643500 10438 raft_consensus.cc:2889] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:54.630381 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.630395 11215 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:54.632906 10585 raft_consensus.cc:2889] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:54.541606 13592 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:10:54.555574 12659 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.649564 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.650352 10581 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
W20260709 12:10:54.653710 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.658668 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.613612 13595 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.662793 10587 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:54.639216 13481 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:54.670343 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:54.637246 13586 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.675169 10437 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [6.3-7.4]
I20260709 12:10:54.652647 13392 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:54.688318 10579 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:54.642596 13214 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:54.653584 13387 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.691849 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.661101 13381 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:54.708609 10510 raft_consensus.cc:2410] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:54.711208 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:54.643867 13506 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.709309 13381 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:10:54.677877 13481 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:54.666811 13571 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.686411 13583 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.725517 13564 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:54.723299 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:54.711035 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.741530 10437 raft_consensus.cc:2410] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:54.736889 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.653595 13584 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.744874 10553 leader_election.cc:400] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6.
I20260709 12:10:54.745306 10553 leader_election.cc:403] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:54.719060 13511 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.709161 13387 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:54.745815 13594 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:10:54.674679 13595 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.770898 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.751216 13564 consensus_queue.cc:1243] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:54.750151 13585 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:54.783870 13387 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:54.787060 13605 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:54.784046 13506 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:54.776430 13416 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.744517 13583 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.740558 13571 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.769085 13477 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:54.716981 13586 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.780890 11218 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:54.776463 13598 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:10:54.815771 13387 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:54.815888 13606 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6.
I20260709 12:10:54.816589 13393 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.816797 13584 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:54.818079 13599 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:10:54.818909 12659 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:54.879623 11220 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:54.896670 10433 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:54.873147 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 384) took 2006 ms. Trace:
I20260709 12:10:54.900825 11212 rpcz_store.cc:276] 0709 12:10:52.866559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:52.866742 (+   183us) service_pool.cc:224] Handling call
0709 12:10:54.872805 (+2006063us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400479fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:54.873016 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600860032}
W20260709 12:10:54.892627 13577 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.860s	user 0.004s	sys 0.006s
W20260709 12:10:54.907778 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 136978us
W20260709 12:10:54.870076 13495 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.701s	user 0.015s	sys 0.000s
W20260709 12:10:54.861672 13573 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.922s	user 0.009s	sys 0.008s
W20260709 12:10:54.912655 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66975.4us
I20260709 12:10:54.869473 13480 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:54.905444 13602 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:10:54.923215 10584 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:54.923475 13456 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.001s
I20260709 12:10:54.924815 13614 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:54.925024 13214 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.927225 10582 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:54.927728 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:54.928809 13584 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:54.929375 13584 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:54.930388 13584 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.932458 10583 raft_consensus.cc:2410] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:54.914582 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:54.848999 13506 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:54.906692 13393 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.942200 13393 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:54.946144 13481 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.920746 13586 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.921989 13595 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.935763 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:54.957010 13576 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
I20260709 12:10:54.962836 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.963740 10505 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:54.965305 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:54.868993 13571 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:54.870671 13374 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:54.853616 13583 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:55.002024 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.851058 13387 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:54.862002 13392 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:54.916790 13495 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.952288 13122 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:54.937453 13573 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.966652 13511 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.969590 13604 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:54.970482 13416 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.975759 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1983) took 1073 ms. Trace:
I20260709 12:10:55.016010 10431 rpcz_store.cc:276] 0709 12:10:53.902152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.087840 (+185688us) service_pool.cc:224] Handling call
0709 12:10:54.114767 (+ 26927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:54.368971 (+254204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.369008 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:54.402340 (+ 33332us) log.cc:844] Serialized 26 byte log entry
0709 12:10:54.613359 (+211019us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:54.647379 (+ 34020us) log.cc:844] Serialized 14 byte log entry
0709 12:10:54.854466 (+207087us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b1800314140. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:54.975185 (+120719us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b1800314140. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c926874bc 00007f2c8f9d548b 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c92ad7060 00007f2c92ae9716 00007f2c92afa287
0709 12:10:54.975368 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.975447 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:54.975493 (+    46us) raft_consensus.cc:1685] finished
0709 12:10:54.975509 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.975682 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121759,"prepare.queue_time_us":28823,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":146088,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":7193,"spinlock_wait_cycles":17285888,"thread_start_us":496017,"threads_started":3,"wal-append.queue_time_us":268729}
I20260709 12:10:54.980386 13567 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:54.981405 13481 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:54.984653 13493 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:54.936443 11218 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:10:54.905130 13585 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:54.986140 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2071) took 1673 ms. Trace:
I20260709 12:10:54.990561 10584 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:55.038010 10586 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:54.990967 13506 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.035198 10507 rpcz_store.cc:276] 0709 12:10:53.312364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.346184 (+ 33820us) service_pool.cc:224] Handling call
0709 12:10:53.346594 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:53.349434 (+  2840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.349487 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:53.350272 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 12:10:53.760815 (+410543us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b18002df820. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:53.842437 (+ 81622us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:53.853181 (+ 10744us) log.cc:844] Serialized 14 byte log entry
0709 12:10:53.853959 (+   778us) log.cc:844] Serialized 14 byte log entry
0709 12:10:53.854170 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:53.854250 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:53.969952 (+115702us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b18002df820. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240
0709 12:10:54.278064 (+308112us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b18002df820. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1
0709 12:10:54.279916 (+  1852us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b18002df820. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:54.509119 (+229203us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b18002df820. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:54.956292 (+447173us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:54.985891 (+ 29599us) raft_consensus.cc:1685] finished
0709 12:10:54.985919 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.986068 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":211,"prepare.queue_time_us":438119,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":341,"raft.run_cpu_time_us":605,"raft.run_wall_time_us":10631,"spinlock_wait_cycles":56747392,"thread_start_us":81333,"threads_started":1,"wal-append.queue_time_us":517117}
I20260709 12:10:54.994732 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:55.043715 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49107.2us
W20260709 12:10:54.995993 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 81513.4us
I20260709 12:10:55.000254 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1966) took 1778 ms. Trace:
I20260709 12:10:54.997742 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.044718 10429 rpcz_store.cc:276] 0709 12:10:53.222248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.474268 (+252020us) service_pool.cc:224] Handling call
0709 12:10:53.494950 (+ 20682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:53.793420 (+298470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.999981 (+1206561us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540034b054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:55.000180 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680342784}
I20260709 12:10:55.045470 10433 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.046247 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.004196 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 269) took 1719 ms. Trace:
W20260709 12:10:55.048487 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 351592us
I20260709 12:10:55.002171 13586 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.048771 10435 rpcz_store.cc:276] 0709 12:10:53.284720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.658458 (+373738us) service_pool.cc:224] Handling call
0709 12:10:53.658900 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:53.700835 (+ 41935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.003796 (+1302961us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b54003a98d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:55.004079 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94667008}
I20260709 12:10:55.049567 13586 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.003443 13474 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.051406 13474 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.007156 13595 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.011075 13214 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:55.041391 11218 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 4
I20260709 12:10:54.998644 13393 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:55.056092 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.051869 13586 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.053397 11218 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.5
I20260709 12:10:55.057639 11218 raft_consensus.cc:2917] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Aborting config change with OpId 6.5: Aborted: Op aborted by new leader
I20260709 12:10:55.058095 11218 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:55.054507 13589 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:55.021735 13387 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.059440 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.063912 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:55.019546 13567 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:55.054996 13493 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.067616 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.068992 10433 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:10:55.033644 13477 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.071169 13616 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:55.073041 10432 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:10:55.019852 13511 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.080338 13387 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:55.023919 13604 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.125983 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:55.127357 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:55.128175 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:55.092283 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.114115 10432 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 4
I20260709 12:10:55.032356 13416 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.127442 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:55.129355 10432 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.5
I20260709 12:10:55.129827 10432 raft_consensus.cc:2917] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Aborting config change with OpId 6.5: Aborted: Op aborted by new leader
I20260709 12:10:55.130456 10432 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:55.068921 13607 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:55.114271 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46805.3us
I20260709 12:10:55.065066 13589 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.110736 13583 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.142005 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:55.060989 13595 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.135975 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.150501 13573 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:55.119158 13612 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.300s	user 0.002s	sys 0.000s
I20260709 12:10:55.119751 13381 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:55.146103 13589 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:55.151293 13607 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.113920 13471 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:10:55.154175 13583 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.115854 13477 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.113467 13493 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:55.192572 13617 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:10:55.209161 13608 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:55.214109 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 78264.9us
I20260709 12:10:55.212193 13381 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.243166 13381 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:55.224777 10580 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:55.232520 13477 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.234412 13583 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:55.247172 13619 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:55.251235 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.234313 13511 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.259148 13392 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:55.261651 13511 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:55.262359 13511 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.228758 13495 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.266630 13511 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:55.266824 10578 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
W20260709 12:10:55.269373 13579 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.035s	user 0.005s	sys 0.005s
I20260709 12:10:55.266590 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:55.282562 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.286108 13618 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:10:55.287379 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 1973) took 1392 ms. Trace:
I20260709 12:10:55.289832 10436 rpcz_store.cc:276] 0709 12:10:53.894878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.895403 (+   525us) service_pool.cc:224] Handling call
0709 12:10:53.895962 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:53.908650 (+ 12688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.908695 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:53.909289 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:10:54.029949 (+120660us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:54.030018 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.030133 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.268702 (+1238569us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:10:55.278915 (+ 10213us) raft_consensus.cc:1685] finished
0709 12:10:55.278942 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.287274 (+  8332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12203,"raft.queue_time_us":12377,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":116,"spinlock_wait_cycles":19848192,"thread_start_us":119804,"threads_started":1,"wal-append.queue_time_us":228598}
I20260709 12:10:55.301358 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.294929 13122 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:55.301115 13393 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:55.298583 13416 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.255366 13608 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.332185 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.334170 10583 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:55.293062 13475 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:10:55.306802 13615 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:10:55.316844 13573 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:55.303786 13392 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:10:55.312096 13506 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:55.357187 13506 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:10:55.357940 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75323.9us
I20260709 12:10:55.360841 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:55.352862 13416 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.369040 11216 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:10:55.357631 13506 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:55.379555 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:55.385360 11216 pending_rounds.cc:85] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 6
I20260709 12:10:55.385735 11216 pending_rounds.cc:107] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:10:55.385998 11216 raft_consensus.cc:2889] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:55.395403 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:55.396096 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.401714 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.402679 10436 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:55.403966 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.405165 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.405623 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.406392 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.407133 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.407230 10437 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:55.407471 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.407904 10438 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:55.408123 10436 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:55.409560 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.367362 13471 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:10:55.389715 13608 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.412246 13543 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:10:55.412286 13495 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.362249 13393 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:55.414378 13495 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:55.415175 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.415454 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:55.416549 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.432273 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.436712 12659 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.440129 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.440963 10429 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:10:55.446234 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.411355 13585 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.422302 13584 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:55.460028 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.423573 11213 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:55.425303 13543 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:55.466624 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.471858 10436 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:55.473150 10579 raft_consensus.cc:2410] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:55.474303 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.467454 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.468539 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.475509 10582 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:55.471257 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.471832 11220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 434) took 1573 ms. Trace:
I20260709 12:10:55.476084 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.476437 11220 rpcz_store.cc:276] 0709 12:10:53.886642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.886933 (+   291us) service_pool.cc:224] Handling call
0709 12:10:54.876548 (+989615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:54.899405 (+ 22857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.459976 (+560571us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b5400479fd0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:55.460122 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274144128}
I20260709 12:10:55.476406 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.480154 10579 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:55.458014 13438 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.406683 13571 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:10:55.416376 13416 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.460377 13608 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:55.462200 11213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 452) took 1067 ms. Trace:
I20260709 12:10:55.483028 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.486577 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 266) took 1277 ms. Trace:
I20260709 12:10:55.486958 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.423866 13212 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 12:10:55.470592 13572 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:55.490175 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.495180 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:55.477756 13471 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.516898 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.497063 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.496809 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.500599 10582 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.470705 13585 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.483800 13633 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.534286 13633 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.536557 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1966) took 1018 ms. Trace:
I20260709 12:10:55.493593 13495 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.536947 13416 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.499342 13511 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.537779 13511 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.423621 13437 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:10:55.493575 13122 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:55.539770 13122 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.540237 13122 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.540839 13511 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.542080 13381 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.517910 10579 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:55.544731 13595 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.544873 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.545877 10579 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:55.499544 13477 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.548031 13477 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.550423 13477 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.488785 13438 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.552943 13438 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:55.553534 13517 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:55.494086 11213 rpcz_store.cc:276] 0709 12:10:54.394859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.401189 (+  6330us) service_pool.cc:224] Handling call
0709 12:10:55.415309 (+1014120us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:55.462126 (+ 46817us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8943488}
I20260709 12:10:55.474900 13506 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.556826 13506 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.559365 13471 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.561542 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.561906 13471 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:55.563603 10582 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:55.564741 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.521660 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.522455 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.524312 10513 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:55.565965 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.525031 10512 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:55.568251 13636 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:10:55.568377 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46812.4us
I20260709 12:10:55.569382 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.570515 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.570546 10583 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:10:55.572976 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.506959 13214 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:55.575423 13214 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.575914 13214 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.577373 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:55.578121 11220 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:55.511269 13572 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.581054 13572 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:55.507513 13635 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:55.533628 13585 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.581745 13635 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.582001 13585 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:55.582232 13635 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:55.582582 13585 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.583122 13585 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.583120 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.583945 10582 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:55.585263 13493 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.536675 13633 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.458199 12659 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.587091 10404 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.588228 13480 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:55.564796 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.588743 13480 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.494966 10508 rpcz_store.cc:276] 0709 12:10:54.208735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.318269 (+109534us) service_pool.cc:224] Handling call
0709 12:10:54.341303 (+ 23034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:54.343206 (+  1903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.486391 (+1143185us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b5400152550. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:55.486516 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51982720}
I20260709 12:10:55.589299 10581 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:10:55.590214 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:10:55.591548 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 68059.9us
I20260709 12:10:55.527849 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 1976) took 1001 ms. Trace:
I20260709 12:10:55.595373 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.546916 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.596379 10582 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:10:55.596948 10579 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:55.597734 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.597754 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:55.600098 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77779.2us
I20260709 12:10:55.603904 10582 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:55.603999 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.605865 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:55.606500 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.606551 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.537132 10584 rpcz_store.cc:276] 0709 12:10:54.518398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.519620 (+  1222us) service_pool.cc:224] Handling call
0709 12:10:55.536353 (+1016733us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b540010ccd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:55.536482 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925418880}
I20260709 12:10:55.609985 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.610236 13381 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:55.616503 12659 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:55.639603 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.640625 10584 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:55.617470 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.617173 13589 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.589294 13480 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.619100 10582 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:55.594627 10578 rpcz_store.cc:276] 0709 12:10:54.526589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.528326 (+  1737us) service_pool.cc:224] Handling call
0709 12:10:55.527598 (+999272us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400276fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:55.527765 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696326400}
I20260709 12:10:55.562911 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:55.621341 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.651258 13589 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.651746 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:55.639297 13591 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.653815 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.654745 13495 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:55.655567 13495 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:55.656054 13495 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.656546 13495 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.633538 13477 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.657940 13477 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.660437 13477 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:55.635946 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:55.636147 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:55.670958 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.647337 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:55.633550 13572 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:55.651986 13589 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:10:55.672194 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65773.7us
I20260709 12:10:55.685789 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:55.524721 13474 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:55.637804 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47683.7us
I20260709 12:10:55.645843 10583 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:55.671284 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.618320 13493 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.632858 13511 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:10:55.763463 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92304.5us
I20260709 12:10:55.764705 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.639420 13438 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:55.639556 13595 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:55.696077 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1983) took 1054 ms. Trace:
I20260709 12:10:55.697185 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:10:55.784816 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87731.2us
I20260709 12:10:55.785946 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:55.793808 10579 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:10:55.777437 13618 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 12:10:55.793004 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 302894us
I20260709 12:10:55.680460 13381 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:55.780052 10580 rpcz_store.cc:276] 0709 12:10:54.641603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.646013 (+  4410us) service_pool.cc:224] Handling call
0709 12:10:55.224211 (+578198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:55.246263 (+ 22052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.695878 (+449615us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400334850. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:55.696010 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322228096}
I20260709 12:10:55.802956 13595 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.808718 13595 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:55.812170 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 391111us
I20260709 12:10:55.680558 13589 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:10:55.748390 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 62785us
I20260709 12:10:55.813103 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.824317 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:55.826570 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62001.3us
I20260709 12:10:55.827527 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.754957 13511 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.755327 13572 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:10:55.736474 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 106948us
I20260709 12:10:55.782500 13523 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:55.700397 13456 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 12:10:55.805116 13584 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:55.822194 13593 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:55.827337 13603 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:55.830061 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.830912 13429 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:55.832712 13477 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:55.844889 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:55.828733 13381 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:55.834755 13511 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.830171 13591 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:10:55.831223 13639 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:55.838501 13523 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.841845 13593 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.846105 13603 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.859616 13586 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:55.858222 13477 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:55.876578 11177 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.809s (SENT) [suppressed 21 similar messages]
I20260709 12:10:55.860808 13608 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.840672 13392 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:10:55.847376 13543 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:55.878659 13603 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:55.905018 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60219.8us
I20260709 12:10:55.869958 13639 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:55.907737 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.874444 13523 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:55.876439 13593 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:55.872416 13474 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:55.910063 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.915715 10437 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [6.3-7.4]
I20260709 12:10:55.878537 13586 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.903928 13477 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:55.925747 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.926301 13474 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.934589 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.937207 10438 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:55.939831 13480 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.943657 13480 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:55.946753 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:55.947813 10437 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:10:55.913765 13639 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.946204 13381 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.946086 13429 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:55.925120 13586 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:55.965668 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:55.875605 13589 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:55.968329 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:55.991773 10433 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.972872 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:55.993676 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 87763.1us
I20260709 12:10:55.950361 13640 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:55.999317 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:55.909403 13584 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:55.988003 13456 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.987035 13645 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:56.011462 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:55.996837 13438 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:10:56.013660 13628 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.470s	user 0.006s	sys 0.005s
I20260709 12:10:56.013384 13429 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.016103 13214 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.018653 13474 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:55.971877 13480 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:10:56.022183 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1578) took 1409 ms. Trace:
I20260709 12:10:56.022786 13618 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:55.990131 13543 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.029740 13523 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.032748 13593 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.035633 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:56.036413 13456 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:56.052258 13645 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:56.018484 13122 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.004633 13640 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:55.959105 13495 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:56.052387 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44238.3us
I20260709 12:10:56.057330 10433 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:56.094322 13652 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:56.068812 10438 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:10:56.065589 10430 rpcz_store.cc:276] 0709 12:10:54.612582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.638943 (+ 26361us) service_pool.cc:224] Handling call
0709 12:10:54.639414 (+   471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:54.647317 (+  7903us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:54.954097 (+306780us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b1800416f20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c969b358f 00007f2c8f9f2571 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd
0709 12:10:55.257282 (+303185us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b1800416f20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:55.383146 (+125864us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b1800416f20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c964d73c7 00007f2c92ae99a7 00007f2c92af90bf 00007f2c92af9038
0709 12:10:55.678483 (+295337us) log.cc:844] Serialized 14 byte log entry
0709 12:10:55.869084 (+190601us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:55.903957 (+ 34873us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.904194 (+   237us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:55.914853 (+ 10659us) log.cc:844] Serialized 14 byte log entry
0709 12:10:55.951631 (+ 36778us) log.cc:844] Serialized 14 byte log entry
0709 12:10:55.955123 (+  3492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.955222 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.021846 (+ 66624us) raft_consensus.cc:1685] finished
0709 12:10:56.021881 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.022088 (+   207us) inbound_call.cc:177] Queueing success response
0709 12:10:56.023948 (+  1860us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b1800416f20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1
0709 12:10:56.024708 (+   760us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b1800416f20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
Metrics: {"mutex_wait_us":4384,"prepare.queue_time_us":41895,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":6879,"raft.queue_time_us":62156,"raft.run_cpu_time_us":566,"raft.run_wall_time_us":35726,"spinlock_wait_cycles":71730816,"thread_start_us":352081,"threads_started":2,"wal-append.queue_time_us":156993}
I20260709 12:10:56.098551 13607 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.072854 11215 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:56.051550 13438 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:56.099846 13438 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:56.037470 13635 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.104740 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.094676 13495 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.053031 13595 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.113377 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.088353 13640 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.088982 13645 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:56.120044 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.109663 13474 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:56.084334 13618 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:56.123245 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 128597us
I20260709 12:10:56.125756 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:56.113335 13214 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.139387 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:56.142459 10438 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:56.138410 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:56.145651 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:56.125805 13585 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.005s
I20260709 12:10:56.141338 11217 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:56.088706 13603 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.141006 13607 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:56.153301 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:56.123886 13474 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:56.159895 10438 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:10:56.161698 11218 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:10:56.161860 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.167601 13603 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.154111 13635 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.157895 13474 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.158924 13607 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.185557 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.156531 13653 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:56.190569 13607 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.191084 13653 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:56.188244 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.174268 13633 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:10:56.159024 13589 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:56.146085 13511 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.154485 13585 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.201136 13585 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.201841 13633 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:56.203079 13633 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.205266 13511 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:56.207347 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:56.208019 11220 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:56.201136 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 1962) took 1690 ms. Trace:
I20260709 12:10:56.212944 10587 rpcz_store.cc:276] 0709 12:10:54.510857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.511039 (+   182us) service_pool.cc:224] Handling call
0709 12:10:54.537051 (+ 26012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:54.665776 (+128725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.200876 (+1535100us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54001a13d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:56.201058 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288044800}
W20260709 12:10:56.217247 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64057.8us
I20260709 12:10:56.217967 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.191780 13653 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:56.221071 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.183573 13613 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:56.225276 11220 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:10:56.183573 13456 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.180233 13656 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:56.182709 13517 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:56.212180 13607 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:56.239864 13653 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.246718 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.226698 13649 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:56.226874 13613 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:56.260244 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.262956 10582 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:56.263408 10586 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:56.263936 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:56.265427 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.266091 10585 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:56.267560 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.276816 10582 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
W20260709 12:10:56.277904 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55297.8us
I20260709 12:10:56.246073 13607 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:56.232560 13438 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.227398 13437 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:10:56.252849 13662 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:56.244899 13593 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.284358 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.309705 10578 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:56.300426 10585 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:10:56.285122 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:56.313235 10587 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:56.288699 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.314517 10586 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:56.253990 13523 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.319903 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:56.289407 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.321025 10584 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.249320 13122 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.251143 13656 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:56.285388 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:56.324738 10579 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:56.292321 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.280905 10580 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:56.261149 13473 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.218966 13589 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.267299 13480 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:56.315738 13618 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.292498 13585 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:10:56.350251 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:56.249464 13517 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:56.348524 10578 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:10:56.314327 13662 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:56.346144 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55577.8us
W20260709 12:10:56.362177 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 69978.4us
I20260709 12:10:56.377385 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.378473 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.355974 13392 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:56.391285 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:56.364322 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104510us
I20260709 12:10:56.294219 13607 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:56.369146 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:56.380731 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:56.397634 10430 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [6.3-7.4]
I20260709 12:10:56.304936 13474 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.386008 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:56.399227 10437 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:56.365065 10584 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:56.397565 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:56.392130 10434 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.354768 13604 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.367596 13523 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:56.413331 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:56.414216 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:56.419268 10438 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:56.399487 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:56.390949 13214 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:56.395874 13493 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:56.416498 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:56.427017 13214 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.427299 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:56.377895 13662 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.434445 11215 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:56.429031 11216 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:56.433821 10429 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:56.433061 10432 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 4.
I20260709 12:10:56.437413 10432 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:56.438642 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.391646 13495 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:10:56.454082 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54744us
I20260709 12:10:56.454115 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:56.440398 10433 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:56.377621 13664 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:10:56.394757 10579 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:10:56.367630 13498 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:10:56.375248 13480 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.464561 10438 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:10:56.363258 13473 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.361900 13613 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:56.479791 13669 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:10:56.480892 13473 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.481088 13476 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:10:56.481564 13663 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:10:56.468039 10433 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:10:56.469133 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:56.429419 13506 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:56.440531 13523 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:56.488876 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:56.489169 13523 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.473413 13438 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.473632 13517 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.435632 13493 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:56.434818 13667 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:10:56.476349 13474 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:56.478775 13480 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.304126 13567 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.483311 10433 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:56.489722 10584 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:56.471715 10434 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:56.493180 10586 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:10:56.471516 10438 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:56.411823 13645 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.486351 12659 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:56.498517 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:56.523917 10579 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:10:56.468358 13657 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:10:56.500545 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 103069us
I20260709 12:10:56.482924 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:56.516186 10431 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:56.486394 13473 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:56.469831 13437 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:10:56.492594 13506 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:56.496140 13604 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:56.532577 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:56.492671 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:56.470083 13668 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:10:56.536571 10507 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:56.513115 13493 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.543663 13567 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:56.582034 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 49596.2us
I20260709 12:10:56.586804 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:10:56.584949 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92356.3us
I20260709 12:10:56.531306 13438 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.543290 11217 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:56.607424 13438 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.611042 13675 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:56.579667 13646 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:10:56.544422 10581 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:10:56.618500 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:56.592948 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:56.627607 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:56.590214 13473 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:56.610308 13438 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.598369 13604 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:56.614285 13589 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.664425 13258 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:56.665122 13258 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:56.665606 13258 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.666316 13585 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.667012 13585 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.615273 13664 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:56.589453 13653 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:10:56.495357 13656 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.639276 13657 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:56.677956 12659 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:56.668471 13438 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.679414 13438 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.682083 13438 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.686059 13657 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:56.639397 13649 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:56.649278 13517 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:56.673076 13480 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:10:56.715732 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 86487.7us
I20260709 12:10:56.722589 10431 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:56.673599 13495 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:56.724987 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.713578 13392 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:56.725970 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.727557 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.728066 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.710641 13214 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.729921 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.736774 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.669214 13437 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 12:10:56.734625 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 102685us
I20260709 12:10:56.734920 13687 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.709836 13657 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:56.739998 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:56.740924 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.723028 13480 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.685647 13506 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.745628 13604 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:56.746439 13586 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.745s	user 0.004s	sys 0.009s
I20260709 12:10:56.771407 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.772058 13495 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.772826 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.729760 13682 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:56.748308 13392 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.749755 13476 consensus_queue.cc:1059] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:56.735932 13671 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:10:56.735846 13630 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 1.223s	user 0.003s	sys 0.011s
I20260709 12:10:56.750530 13607 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.790819 13607 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:56.778757 13477 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.831s	user 0.015s	sys 0.000s
I20260709 12:10:56.791102 10430 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:56.750563 13640 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.635s	user 0.012s	sys 0.000s
I20260709 12:10:56.762457 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.791586 13477 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.791826 13640 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.792892 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.793895 13477 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.794096 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.794641 10431 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:56.794739 10512 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:56.767397 13480 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:56.768826 13667 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:56.713338 13649 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.781812 13678 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:56.797616 13678 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:56.797997 13678 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.765291 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:56.774068 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.717056 13517 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.803574 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.804750 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.777830 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.711318 13664 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.788237 13476 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:56.808349 13476 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:56.809362 13476 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.720372 13473 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.810230 13517 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.760080 13687 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.802268 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.814599 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.815625 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.824386 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.825572 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.826395 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.831411 10511 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:56.830454 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.833473 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.834966 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.827826 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.837121 10508 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:56.839838 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.771415 13586 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.793900 13640 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.813337 13480 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:56.846141 10505 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:56.862125 13687 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:56.867233 13644 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:56.869180 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:56.871301 13441 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:56.872251 13441 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:56.873359 13441 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:56.874471 13441 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:56.839943 13585 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.875128 13480 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:56.876664 13480 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.868961 13644 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:56.874927 13441 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:56.893289 10511 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:56.897675 10432 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:56.901525 13685 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:56.902262 13667 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:56.903733 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1661) took 2855 ms. Trace:
I20260709 12:10:56.904124 13586 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:56.906843 13639 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.915s	user 0.000s	sys 0.013s
I20260709 12:10:56.907397 13639 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.908553 10513 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:56.909718 10504 raft_consensus.cc:1217] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-6.5]   Dedup: 6.4->[6.5-6.5]
I20260709 12:10:56.910297 10504 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [6.4-6.5]
I20260709 12:10:56.904191 10509 rpcz_store.cc:276] 0709 12:10:54.048729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.050871 (+  2142us) service_pool.cc:224] Handling call
0709 12:10:54.051517 (+   646us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:54.142136 (+ 90619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.142230 (+    94us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:54.143433 (+  1203us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.804298 (+1660865us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b180033e3e0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:10:56.002276 (+197978us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:56.002317 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.002389 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.902536 (+900147us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b2400167558. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa30fb0 00007f2c926a7488 00007f2c926a7be8 00007f2c9267f398 00007f2c9267a76a 00007f2c92679bb6
0709 12:10:56.903472 (+   936us) raft_consensus.cc:1685] finished
0709 12:10:56.903497 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.903661 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89879,"raft.queue_time_us":91629,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7848,"spinlock_wait_cycles":68329984,"thread_start_us":197722,"threads_started":1,"wal-append.queue_time_us":1965857}
I20260709 12:10:56.892763 13441 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:56.913617 13441 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:56.915778 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:56.917896 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2078) took 2452 ms. Trace:
I20260709 12:10:56.919588 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.920847 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.921947 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.917976 13685 consensus_queue.cc:1243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:10:56.918649 13690 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:56.923288 13650 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:56.940932 13650 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:56.918520 10504 rpcz_store.cc:276] 0709 12:10:54.465184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.466118 (+   934us) service_pool.cc:224] Handling call
0709 12:10:56.909309 (+2443191us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:56.915176 (+  5867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.917647 (+  2471us) spinlock_profiling.cc:243] Waited 2.41 s on lock 0x7b540036e050. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:56.917819 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5551358720}
I20260709 12:10:56.942132 10505 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:56.924503 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.924959 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.936084 10511 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:56.945420 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.947800 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.950637 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:56.952801 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.953004 13657 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.954025 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.956157 13639 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:56.942592 13586 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:56.963476 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 65969us
I20260709 12:10:56.965471 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.923448 13690 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:56.968415 13641 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:56.968629 13685 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:56.968895 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:56.969314 13694 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:10:56.969483 13649 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:56.970486 13667 raft_consensus.cc:1064] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:56.979768 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:56.980597 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.975812 13662 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:56.998433 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:56.980185 13690 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:56.958770 13586 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.003325 10438 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:57.003829 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:57.007920 10506 raft_consensus.cc:606] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:56.982203 13641 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:57.014894 10504 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:57.015445 10504 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.018709 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1594) took 1573 ms. Trace:
I20260709 12:10:57.020498 13586 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.019209 10435 rpcz_store.cc:276] 0709 12:10:55.445807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.457297 (+ 11490us) service_pool.cc:224] Handling call
0709 12:10:55.475607 (+ 18310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:55.478390 (+  2783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.478426 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:55.480068 (+  1642us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.831425 (+351357us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:55.859208 (+ 27783us) log.cc:844] Serialized 14 byte log entry
0709 12:10:55.859563 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.859648 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.970820 (+1111172us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b180048bda0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:10:57.018358 (+ 47538us) raft_consensus.cc:1685] finished
0709 12:10:57.018486 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:57.018634 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2290,"prepare.queue_time_us":12500,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":4154,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":6008,"spinlock_wait_cycles":4907264,"thread_start_us":335207,"threads_started":1,"wal-append.queue_time_us":389199}
I20260709 12:10:57.022688 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:57.024006 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.024623 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.025149 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.026437 13572 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:57.027335 13572 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.029830 13645 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.029115 13641 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.030150 13595 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.032282 13657 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.030611 13572 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:57.034338 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.034864 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:57.035504 10435 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.024525 13613 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:56.986874 13650 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.042733 13650 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.052145 10504 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.038044 11213 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:10:57.052983 13678 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:57.078138 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:57.079031 10506 raft_consensus.cc:606] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:57.119560 13258 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.089454 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:57.123782 13678 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:57.124596 10438 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.143136 10504 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.156862 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:57.157819 10587 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
W20260709 12:10:57.159164 13493 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.002s
I20260709 12:10:57.159415 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:57.160579 10580 raft_consensus.cc:2410] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:57.159813 13493 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.161744 10437 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:57.161639 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.162371 10582 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:57.153096 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.154127 10585 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:57.143735 10505 raft_consensus.cc:2410] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:10:57.168182 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:57.168905 10510 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:57.172243 10582 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:57.172878 10585 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:57.173306 10438 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:57.173728 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.174185 10585 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:57.174907 10438 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:57.176139 10585 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:57.176457 10438 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:57.177877 10438 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:57.148895 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:57.179891 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.180642 10581 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:57.182224 10587 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:10:57.183256 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.152060 10435 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:57.186134 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.186944 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:57.192664 13608 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:57.199558 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.201224 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c882875c377649acb233035d47b8132e"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:57.201869 10506 raft_consensus.cc:606] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:57.202204 10506 raft_consensus.cc:614] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:57.193909 13608 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.206779 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.207535 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:57.208315 10587 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:57.207542 10511 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:57.149267 13615 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:57.216356 10504 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:10:57.216832 10504 pending_rounds.cc:85] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 7
I20260709 12:10:57.217150 10504 pending_rounds.cc:107] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:10:57.198082 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:57.217459 10504 raft_consensus.cc:2889] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:57.220345 13615 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.221863 10579 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:57.225071 13660 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:57.226266 13660 consensus_queue.cc:1243] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 10.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:57.227923 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:57.228502 13608 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.229449 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:57.237469 10579 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:10:57.237687 10584 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:57.240409 10579 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:57.226970 13660 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:10:57.246204 13660 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.251261 13644 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:57.253201 13685 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:57.253767 13685 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.209267 13681 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:57.212790 13441 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:10:57.256234 13441 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:57.257902 13639 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:57.258356 13639 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.259667 13639 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.262167 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1688) took 1153 ms. Trace:
I20260709 12:10:57.262621 10512 rpcz_store.cc:276] 0709 12:10:56.108926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:56.109159 (+   233us) service_pool.cc:224] Handling call
0709 12:10:56.109508 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:56.795221 (+685713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:57.261847 (+466626us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b5400197dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:57.262085 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1575440768}
I20260709 12:10:57.263947 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.264616 10512 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:57.265530 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.273109 13258 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.273348 13640 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:57.273943 13640 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.278617 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.280575 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:57.281713 10581 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:10:57.285058 10512 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:57.285420 10580 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:57.258584 13662 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.291590 10579 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:10:57.296451 13688 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:57.297957 10581 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:10:57.302784 10579 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:10:57.293061 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.307533 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.208163 13673 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 12:10:57.318038 10432 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:57.319278 13572 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.320016 13572 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.321940 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.323273 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:57.323602 13456 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:57.324420 13456 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.324484 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.329154 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.333537 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:57.288117 13656 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:57.339057 10512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:10:57.215086 13667 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.344687 13690 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.299254 13639 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.347836 13572 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:57.350723 13673 consensus_queue.cc:1243] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:57.351572 13673 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:57.352658 13673 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:57.354015 13673 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:57.354462 13697 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:57.355123 13697 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.356880 13670 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:57.356485 13697 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.357460 13670 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.358618 13697 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:57.363206 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.364305 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.341244 10512 raft_consensus.cc:606] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:57.342609 13456 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:10:57.369432 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.371075 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:57.376040 10511 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:57.358897 13670 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.379693 13670 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.381412 10506 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
W20260709 12:10:57.386531 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.389385 13660 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.354494 13673 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.396783 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1691) took 1279 ms. Trace:
I20260709 12:10:57.397688 13690 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.475283 13596 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.476132 13596 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.477053 13677 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:57.398208 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1695) took 1199 ms. Trace:
I20260709 12:10:57.460300 13608 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:57.479215 10508 rpcz_store.cc:276] 0709 12:10:56.198559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:56.199853 (+  1294us) service_pool.cc:224] Handling call
0709 12:10:56.200267 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:56.837567 (+637300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:57.397878 (+560311us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54003fded4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:57.398121 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464145280}
I20260709 12:10:57.479786 13608 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:57.400799 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.486204 10584 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:10:57.405522 13664 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:57.407940 10510 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:57.487586 10510 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.409605 13493 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.340678 13615 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:57.327315 13688 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.310534 13258 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:10:57.340962 13682 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.333451 13640 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.338728 13656 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:57.423887 13567 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.006s
I20260709 12:10:57.428114 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:57.429131 10432 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:57.438403 10511 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:57.460345 10507 rpcz_store.cc:276] 0709 12:10:56.117662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:56.117768 (+   106us) service_pool.cc:224] Handling call
0709 12:10:56.118060 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:56.598192 (+480132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:57.396408 (+798216us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b54002c5bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:57.396692 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961977600}
I20260709 12:10:57.473556 13673 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:57.477913 13677 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:57.488518 10510 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:57.532284 10479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.041s	sys 0.054s
W20260709 12:10:57.581141 10479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.042s	sys 0.059s
I20260709 12:10:57.494613  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:10:57.569085 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.558336 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.493877 10587 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:57.586337 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.588769 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.581830 10510 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:57.581756 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 579980us
I20260709 12:10:57.585498 13618 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:10:57.515878 10322 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:10:57.587396 13613 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.007s
I20260709 12:10:57.545848 13214 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:10:57.599357 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 278497us
I20260709 12:10:57.502657 13660 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:57.602859 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:57.590472 13480 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:57.611060 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.609862 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:57.603485 13673 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.612113 13572 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:10:57.606381 13523 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:57.598140 10322 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 82347.1us
I20260709 12:10:57.607352 13473 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:57.589815 13682 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.617300 13567 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:57.602453 13214 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:57.651571 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.656365 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.626585 13438 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.617880 13480 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.630375 13608 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:57.635354 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 435790us
I20260709 12:10:57.634946 10510 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:57.639396 13567 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:57.678287 13567 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.693104 13641 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:57.637355 13523 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.695149 13608 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.695686 13608 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.697883 13523 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.698166 13608 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:57.701279 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.675853 13493 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.706943 13523 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:57.633149 13690 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:10:57.708475 13677 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.708578 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:57.758513 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50028.9us
I20260709 12:10:57.761031 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:57.589952 13687 consensus_queue.cc:1175] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 8 status { last_received { term: 7 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:10:57.711107 13212 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:57.712224 13688 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:57.712599 13437 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:57.723484 13596 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.671828 13686 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:10:57.730158 13650 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.004s
I20260709 12:10:57.717559 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:10:57.708231 13666 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.345s	user 0.004s	sys 0.000s
I20260709 12:10:57.650807 13258 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 12:10:57.768224 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.762552 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:57.637874 13473 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.769306 12659 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.789033 12659 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.770176 13608 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:57.792001 12659 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:57.792765 13608 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:57.795562 10509 raft_consensus.cc:2410] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:10:57.795523 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:57.798194 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:57.800487 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:57.805150 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 87470.3us
I20260709 12:10:57.790695 13473 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.822099 10586 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:10:57.823313 10578 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:57.822081 10587 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:57.799642 13608 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:10:57.828738 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 204715us
I20260709 12:10:57.827950 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:10:57.637472 13656 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.734301 13618 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.769995 13523 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.807683 13650 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:57.808960 13122 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:57.823374 12659 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:57.804102 13686 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:57.865259 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.849514 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:57.828739 13473 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:10:57.876196 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:57.883193 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.818254 13690 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:10:57.884989 10580 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 8 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:57.826143 13652 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:10:57.851974 13674 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:10:57.895079 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.787688 13214 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.832742 13437 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:10:57.774098 13677 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:57.849562 13656 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:57.906461 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:57.765654 13493 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.897749 13608 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:57.900347 13650 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:10:57.924314 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 123932us
I20260709 12:10:57.888923 13682 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.930260 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:57.941088 13716 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:10:57.949599 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:58.001387 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 65366us
I20260709 12:10:58.003036 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:58.004631 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:57.877091 13122 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:57.894600 13441 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:10:57.917609 13721 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:10:57.924976 13543 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:10:57.951207 13652 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.953828 13437 consensus_queue.cc:1243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:57.891394 12659 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:57.961401 13670 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.009s
I20260709 12:10:57.959568 13677 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:57.968443 13523 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.011709 13719 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:58.019394 13523 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.019608 13437 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:58.020095 13437 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.020156 13677 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:58.021024 13258 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:58.021463 13258 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.021152 13437 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.022642 13437 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.972368 13493 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.928951 13214 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.026265 12659 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:57.974849 13686 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:57.929690 13706 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:58.027410 13706 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.027778 13721 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:58.028226 13523 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:57.976740 13650 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.978142 13608 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:57.978847 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.031559 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1696) took 1823 ms. Trace:
I20260709 12:10:57.982612 13438 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:10:57.983139 13653 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:10:57.881073 13662 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:10:57.934026 13674 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:57.988962 13640 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.939572 13724 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:57.990371 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:57.995294 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.048938 11218 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
W20260709 12:10:58.005911 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.008584 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:58.013041 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:58.050705 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.015324 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.022334 13258 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:57.969375 13712 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.028546 13652 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.029412 13480 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:58.032075 10513 rpcz_store.cc:276] 0709 12:10:56.207764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:56.207925 (+   161us) service_pool.cc:224] Handling call
0709 12:10:56.208260 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:56.908918 (+700658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:58.031277 (+1122359us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54003165d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:58.031491 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1610105728}
I20260709 12:10:58.009218 13673 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:58.028323 13706 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:58.051944 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 218104us
I20260709 12:10:58.028225 13721 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:10:58.052680 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44185.2us
I20260709 12:10:58.016305 13728 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:10:58.066107 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.066694 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.058495 13712 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.073138 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:58.034945 13608 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.056861 13729 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.079322 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:58.061792 13122 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.072240 13258 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.084892 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:58.053354 13567 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.019680 13719 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.094135 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:58.087074 13681 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.036s	sys 0.020s
W20260709 12:10:58.096537 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 189701us
I20260709 12:10:58.097028 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:58.096544 13681 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.037s	sys 0.020s
I20260709 12:10:58.074316 13493 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:58.097780 13681 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:58.098253 13493 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:58.098901 13493 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.099809 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.100484 10432 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:58.100518 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.101650 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.101856 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.102270 10433 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:58.103408 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:58.103572 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 287386us
I20260709 12:10:58.104234 10438 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:58.105631 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:58.106953 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:58.107762 10504 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:10:58.074221 13677 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:58.076327 13706 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.097720 10429 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:58.111631 10434 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:58.071269 13473 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:10:58.068177 13673 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:58.122454 10434 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:58.108451 13662 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:58.114118 13677 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:58.116336 13641 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:58.134610 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.135252 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.121245 13731 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.116500 13596 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:10:58.116849 13727 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:10:58.135456 13677 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.153110 13712 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.139752 13662 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.156886 13662 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.157822 13608 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:58.147085 13731 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.159318 13719 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.160118 13731 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.160983 13493 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.162576 13493 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.153539 13613 raft_consensus.cc:1027] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: attempt to promote peer 52bae2d2caf94d97b1049ecd4f82a2df: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:10:58.155743 13681 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.123832 13721 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.164723 13721 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.151182 13641 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:58.108592 13674 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.173193 13652 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.190584 13677 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.191296 13710 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:10:58.193655 13710 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.244527 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:58.268783 13641 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.252949 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:58.277405 13677 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:58.272189 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:58.284013 13572 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }.
  Pending config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.286834 13727 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.289791 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:58.263747 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.294289 10434 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:58.295496 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.296520 10434 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:58.297662 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:58.297889 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2153) took 1143 ms. Trace:
I20260709 12:10:58.300112 10435 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:10:58.300467 10581 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:58.302506 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:58.306584 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.307376 13392 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:58.308713 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.309525 13714 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.254374 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:58.315943 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.316850 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.324206 10504 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 12:10:58.324694 10504 pending_rounds.cc:85] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 3
I20260709 12:10:58.324987 10504 pending_rounds.cc:107] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 12:10:58.325285 10504 raft_consensus.cc:2889] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:58.291972 10583 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2144) took 2026 ms (client timeout 2019 ms). Trace:
W20260709 12:10:58.328550 10583 rpcz_store.cc:269] 0709 12:10:56.264488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:56.265098 (+   610us) service_pool.cc:224] Handling call
0709 12:10:58.289443 (+2024345us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:10:58.290798 (+  1355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8821888}
I20260709 12:10:58.298537 10434 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:58.298512 10584 rpcz_store.cc:276] 0709 12:10:57.154805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.156482 (+  1677us) service_pool.cc:224] Handling call
0709 12:10:58.297447 (+1140965us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b540010b3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:58.297692 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116946304}
I20260709 12:10:58.333289 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:58.335489 13645 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:10:58.335767 13683 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:58.336300 13683 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.336453 13645 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.337908 10436 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:10:58.339720 10436 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:10:58.339835 13645 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.339988 10511 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:10:58.340516 10511 pending_rounds.cc:85] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 8
I20260709 12:10:58.340787 10511 pending_rounds.cc:107] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:10:58.341128 10511 raft_consensus.cc:2889] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:58.337852 10513 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:10:58.342376 10513 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.344195 10513 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.346212 10513 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:10:58.348623 10509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 281) took 1426 ms (client timeout 1809 ms). Trace:
W20260709 12:10:58.349169 10509 rpcz_store.cc:269] 0709 12:10:56.921659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:56.945139 (+ 23480us) service_pool.cc:224] Handling call
0709 12:10:58.348060 (+1402921us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000f82d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:58.348391 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952893696}
I20260709 12:10:58.347746 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:58.319381 13714 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.338189 13683 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.363399 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2016) took 1076 ms. Trace:
I20260709 12:10:58.363718 13683 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.364655 13429 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:58.288158 13727 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:58.372448 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.374322 13727 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.376118 13476 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:58.379272 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.382972 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:58.383968 10587 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.385903 13714 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:58.387084 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:58.388346 10578 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:58.360059 10436 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:58.370303 13392 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.369215 13429 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.371289 10504 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:10:58.367678 10581 rpcz_store.cc:276] 0709 12:10:57.287001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.319139 (+ 32138us) service_pool.cc:224] Handling call
0709 12:10:58.299618 (+980479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.301156 (+  1538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:58.363033 (+ 61877us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b540010b3d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:58.363228 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2253406080}
I20260709 12:10:58.395972 10504 pending_rounds.cc:85] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 7
I20260709 12:10:58.401222 10504 pending_rounds.cc:107] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:10:58.401528 10504 raft_consensus.cc:2889] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:58.396224 11215 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
W20260709 12:10:58.402922 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.411373 13727 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:58.412325 13727 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.415688 13727 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:58.417655 10429 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:10:58.424026 10432 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:58.426044 13567 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:58.426407 13685 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.427947 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:58.428956 10438 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.429991 13656 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.431499 10586 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:58.431834 13429 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.434062 10579 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
W20260709 12:10:58.428054 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 80870.9us
I20260709 12:10:58.439216 13429 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.443143 13673 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:58.444319 10582 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
W20260709 12:10:58.439883 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.433969 10432 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:10:58.447180 13639 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:10:58.447621 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.449806 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:58.447625 13639 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.452461 13639 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.453539 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:58.454141 13639 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.449879 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:58.455528 10430 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.457846 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:58.461956 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.427976 13685 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.467703 10438 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
W20260709 12:10:58.351346 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 212340us
I20260709 12:10:58.408715 13392 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.475945 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:10:58.480311 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.483721 13690 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.484517 13690 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.486701 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:58.487555 10431 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.488677 13697 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:10:58.490111 13710 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:58.496876 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.498839 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:58.499586 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.507019 10583 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:10:58.508411 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.509169 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.509598 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.513027 13673 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:58.513970 13673 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.515816 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.519097 13673 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.519979 10433 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:58.515017 13690 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.527891 10429 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:58.529685 10432 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:58.527895 10581 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:58.532204 10432 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:58.534709 10511 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:10:58.536772 13653 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:58.537210 13653 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.537483 13690 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.538384 13690 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.540082 13675 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.540818 13675 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.538542 13653 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.543090 13656 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:58.544519 13675 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:58.544281 13653 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:58.545302 13673 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:58.549041 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.550052 10432 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:58.550729 11215 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:58.552152 11220 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:10:58.561180 10578 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:58.553247 13610 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:10:58.554190 13207 consensus_queue.cc:1059] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:58.563980 10578 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:10:58.565052 13214 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.541s	user 0.016s	sys 0.001s
I20260709 12:10:58.578555 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.560384 11218 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:58.579538 13212 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.581190 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.580673 13212 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.583252 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.583994 13670 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.584666 13670 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.565953 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.567301  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:58.556881 13690 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.569518 13207 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:58.574193 10436 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:58.574965 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:58.576828 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.576927 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.556427 13710 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:58.601128 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.603039 11220 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:58.604266 10431 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:58.589617 13207 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:10:58.606645 10434 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:10:58.595439 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:58.612274 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.612480 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:58.613128 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.614952 10430 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:58.620231 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.621003 10438 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:58.621148 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.621997 10434 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:58.622372 10438 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:58.623149 13498 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:10:58.624106  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:58.625408 10438 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:58.630347 10584 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:10:58.619098 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 141956us
I20260709 12:10:58.627429 13677 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:58.567806 13673 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:10:58.634617 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:10:58.643524 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 143812us
I20260709 12:10:58.630120 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.645961 10431 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:58.646883 13656 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:58.647387 10437 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:10:58.648972 10434 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:58.660454 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.660871 13670 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.665016 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.662427 10432 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:58.645088 13690 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:58.678964 10587 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:58.684680 10434 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:58.686631 11220 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:10:58.658346 13677 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:58.673774 13690 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:58.701587 13207 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.703418 13207 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.703745 13613 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:58.704595 13613 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.698534 10581 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:10:58.707331 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.708905 10436 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:58.685451 13683 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.707123 13690 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.693887 13214 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:10:58.732800 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67951.2us
I20260709 12:10:58.735586 13697 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:58.736171 13697 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:58.736359 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.735589 13441 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:58.737836 13441 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.737478 13677 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.736636 13697 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.737317 10430 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:10:58.703424 13660 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:10:58.751164 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.751782 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.749086 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.753736 13214 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.755414 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:58.756294 13214 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.757864 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:58.758708 10582 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:58.759014 11213 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:10:58.759979 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.757193 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:10:58.764436 13640 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.642s	user 0.000s	sys 0.022s
I20260709 12:10:58.759028 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.746490 13593 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:10:58.767011 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:58.767910 13593 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:58.768287 13593 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:10:58.769582 13258 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.676s	user 0.005s	sys 0.009s
I20260709 12:10:58.769477 13593 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:10:58.770437 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.771091 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.739126 13441 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.771951 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.774063 13640 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:58.774838 13660 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.774212 13653 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:58.787802 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.788630 11220 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
W20260709 12:10:58.739529 13437 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.698s	user 0.016s	sys 0.008s
I20260709 12:10:58.790386 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2002) took 2250 ms. Trace:
I20260709 12:10:58.724470 13613 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:58.791082 10437 rpcz_store.cc:276] 0709 12:10:56.530116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.163170 (+633054us) service_pool.cc:224] Handling call
0709 12:10:57.163556 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:58.648012 (+1484456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:58.750108 (+102096us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001676d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:10:58.779282 (+ 29174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3370962688}
I20260709 12:10:58.794219 10587 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:58.774471 10436 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:58.717317 13683 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.736371 13212 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:58.795965 10584 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:10:58.807562 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.779163 13258 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:58.805228 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:58.814926 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.815977 10580 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:10:58.816583 13653 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:58.817587 13473 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:58.819177 13662 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.819880 13662 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:58.823247 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76382.6us
W20260709 12:10:58.828685 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.829499 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:58.829517 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.833676 10432 raft_consensus.cc:2376] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10. Re-sending same reply.
I20260709 12:10:58.839407 13441 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.812439 13258 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:58.818151 13653 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.840267 13258 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:58.844960 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:58.846777 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:58.848302 13476 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:58.850253 13674 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:10:58.853407 10578 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:10:58.853132 13613 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:58.854964 13613 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:58.855536 13613 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:58.777848 13660 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:58.826687 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:58.857304 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:10:58.859477 10513 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.003s
I20260709 12:10:58.862239 10513 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:10:58.864576 10587 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:58.818424 13473 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:58.830860 13593 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.865797 10430 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:58.871595 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:58.862584 10513 pending_rounds.cc:85] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 7
I20260709 12:10:58.876233 10581 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:10:58.873819 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:58.877256 13498 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:10:58.885124 10513 pending_rounds.cc:107] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:10:58.885556 10513 raft_consensus.cc:2889] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:58.879791 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.880300 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.882488 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:58.888114 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.890642 10432 raft_consensus.cc:2410] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:58.890348 10580 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:58.894009 10432 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:58.894483 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:58.895157 13704 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:58.895980 10431 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:58.897717 13473 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:58.899396 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:58.903759 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.905522 10436 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:10:58.908051 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:58.916812 10432 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:10:58.918558 11216 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:10:58.921221 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.923383 10436 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:10:58.924376 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65922us
I20260709 12:10:58.929634 10433 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:10:58.929821 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.886878 13613 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:58.970688 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.977413 13704 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.008571 10437 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
W20260709 12:10:59.013051 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 186528us
W20260709 12:10:59.018275 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 85865.7us
I20260709 12:10:59.023450 10436 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:59.027767 10434 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:10:59.029388 10434 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:10:59.031556 13613 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:59.031996 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.033037 13610 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.039088 13610 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:59.007293 13429 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:59.050246 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 156127us
I20260709 12:10:59.056922 13724 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:59.057662 13724 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.060137 13572 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:59.060447 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:59.061441 13214 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:59.061882 13214 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.062251 13214 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:59.063944 13704 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.065623 10436 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:59.065522 10504 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:10:59.072851 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:10:59.091859 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47608.4us
I20260709 12:10:59.048164 13429 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.078819 10509 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:59.099388 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.103456 11217 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:10:59.100584 13727 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:59.115267 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.121340 13572 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.129603 13572 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.101370 13640 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.077060 13653 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 12:10:59.138573 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70738.2us
I20260709 12:10:59.158428 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:59.122043 13754 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:59.163022 13572 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.134804 13662 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:59.169344 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 91500.8us
I20260709 12:10:59.175827 10437 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:59.180122 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:59.183712 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.184027 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:59.168686 13754 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:59.190274 11215 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:59.193105 13763 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:59.194180 13763 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.195047 13763 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.198225 13763 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:59.202258 13614 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:10:59.161495 13710 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:10:59.150267 13640 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.208866 13495 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:10:59.201377 13754 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:59.215524 10507 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:59.220927 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:59.222450 13429 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.141726 13690 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:59.225654 11217 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:59.247891 10504 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:10:59.228088 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.231495 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:59.236837 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:59.253386 10511 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:59.253820 10509 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:10:59.253835 10512 raft_consensus.cc:2410] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:10:59.258246 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 149839us
I20260709 12:10:59.259390 10507 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:10:59.280026 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:59.281862 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 107620us
I20260709 12:10:59.256629 13662 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.287269 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.257035 13429 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.288026 10430 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:10:59.289063 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 52364.4us
I20260709 12:10:59.289101 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.294185 13653 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.298032 13441 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:59.298769 13653 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.302237 13653 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.302858 13755 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.307310 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:59.307930 13690 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.308139 11216 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:59.318369 10431 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:10:59.319994 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.321776 10430 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:10:59.324599 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:59.325750 13640 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:59.328533 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.334997 13572 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:59.295604 13710 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:10:59.335440 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.337153 13429 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:10:59.338271 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.338871 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.339974 13429 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
W20260709 12:10:59.341867 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 141272us
I20260709 12:10:59.342924 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.338661 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:59.347523 10432 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:10:59.346269 13258 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.283429 13697 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.354732 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.355471 10430 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:59.356487 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.294762 13662 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.365873 13572 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:59.368059 13572 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.363684 13614 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.368459 13572 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:59.359197 13610 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:10:59.361011 13763 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.371593 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:59.371915 13763 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.373101 13697 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.378665 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:59.381376 13724 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.383651 13727 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:59.384284 13727 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:59.385356 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.354069 13473 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:59.387187 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2026) took 1090 ms. Trace:
I20260709 12:10:59.387547 13473 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:59.388043 13473 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:59.387758 10433 rpcz_store.cc:276] 0709 12:10:58.296958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.520936 (+223978us) service_pool.cc:224] Handling call
0709 12:10:58.923937 (+403001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.930259 (+  6322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.386864 (+456605us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54002666d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:10:59.387028 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925868928}
I20260709 12:10:59.392613 13765 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:59.393328 13258 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.394052 13258 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:10:59.397429 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:10:59.398022 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:59.398852 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:59.399632 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:10:59.399698 13640 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:59.401472 13640 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:59.402158 10510 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:10:59.402554 10510 pending_rounds.cc:85] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 6
I20260709 12:10:59.402806 10510 pending_rounds.cc:107] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:10:59.414201 10510 raft_consensus.cc:2889] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:59.404659 13710 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.415411 13710 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.418047 13641 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:59.418906 13662 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.420112 13710 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.398269 10509 raft_consensus.cc:2368] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10.
I20260709 12:10:59.408687 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:59.409766 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:59.407666 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:59.427404 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:10:59.431444 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 88688.1us
I20260709 12:10:59.423017 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:10:59.433568 11217 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:59.433600 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.432246 10508 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:10:59.441815 10512 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:59.441771 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:10:59.449378 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.450503 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:59.461740 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:59.441260 13683 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.465768 13754 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:59.469743 10479 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:59.471652 13653 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:59.472281 13763 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.472865 13763 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.473351 13763 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.475148 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.475519 13639 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:59.475929 13639 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.476348 13639 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.477421 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.479409 13610 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:59.479864 13610 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:59.480285 13610 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:59.479820 13613 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.483705 13764 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.483668 13613 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.484833 13764 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:59.459802 13656 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:59.486955 13613 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:59.482329 13572 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.489123 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:59.495244 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:59.499333 11217 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:10:59.497730 13662 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:59.503852 13662 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.504297 13662 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:59.504393 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:59.489164 13764 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:59.459440 13724 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:59.495359 13613 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:10:59.544399 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 82779.5us
I20260709 12:10:59.486692 13639 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:59.556809 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 160136us
I20260709 12:10:59.562256 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:59.564967 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:59.494772 13683 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 12:10:59.577154 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.525980 13495 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:59.587021 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82725.1us
I20260709 12:10:59.587139 11177 leader_election.cc:400] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10.
I20260709 12:10:59.584301 10509 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:10:59.548293 13441 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:10:59.591639 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:10:59.592491 10437 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:59.594389 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:10:59.548714 13764 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:10:59.552440 13677 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.566792 13704 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:59.595270 10433 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:10:59.603668 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:10:59.602288 13214 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.006s
I20260709 12:10:59.566576 13498 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:10:59.565115 13724 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:10:59.675637 10582 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:59.613132 13763 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:59.588248 13613 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.644091 13429 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:10:59.693706 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 128779us
I20260709 12:10:59.696345 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.614182 13258 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:10:59.640461 13710 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 12:10:59.717363 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:59.715922 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 105937us
I20260709 12:10:59.718307 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:59.720206 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:10:59.649230 13214 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.658908 13690 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:10:59.723062 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:10:59.706195 13763 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:59.727970 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 181598us
I20260709 12:10:59.702589 13572 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:10:59.729480 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 83500.4us
I20260709 12:10:59.730332 13788 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:59.689786 13773 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:10:59.689364 13677 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.732800 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:10:59.739975 13770 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:10:59.745306 13773 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:59.726572 13763 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:10:59.754784 10580 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:10:59.722340 13754 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.769075 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:10:59.748159 13788 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.769874 11221 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:59.746608 13660 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:59.775202 13774 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:10:59.750108 13764 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:59.774204 13768 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:10:59.757947 13763 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:59.779198 13788 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:59.788815 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:10:59.803316 13777 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:10:59.806133 13677 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:10:59.809640 13214 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.810891 13613 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:10:59.812575 13429 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:10:59.814237 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:10:59.818275 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:10:59.818960 13787 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:59.819114 13785 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.819787 13785 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.819479 13787 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.825026 13258 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:59.825673 13258 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:10:59.782296 13773 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:10:59.834762 11221 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:10:59.825021 13787 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:59.837702 13785 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:59.842727 13778 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.848001 13724 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:59.792507 13656 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.775238 13660 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.855703 13790 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:59.876083 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 155985us
I20260709 12:10:59.862150 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:59.878262 13724 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:59.878685 13724 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.813131 13429 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:10:59.784930 13610 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.879036 13724 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:59.788339 13639 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:59.876956 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 152239us
I20260709 12:10:59.883066 13786 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:59.892261 13773 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:10:59.898155 13727 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.001s
I20260709 12:10:59.832101 13258 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:10:59.901732 13787 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.907274 13786 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:59.913895 13786 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:59.916745 10332 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 4 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:59.918510 10332 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:59.927223 13704 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:10:59.927788 13704 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:59.927124 10332 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 8, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 8 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:10:59.929236 10332 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:10:59.930382 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49122.2us
I20260709 12:10:59.931373 10332 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 6 to 8, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 8 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:10:59.881657 13656 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.911973 13765 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.932067 10552 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:10:59.941854 10437 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
W20260709 12:10:59.961968 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 152824us
W20260709 12:10:59.939805 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.933444 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:10:59.943655 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:10:59.898715 13727 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.908919 13787 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:10:59.962509 13473 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.002s
I20260709 12:10:59.992094 13473 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.965044 13765 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:10:59.997825 13765 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:10:59.970438 13572 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.944172 13610 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:00.000025 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 308824us
W20260709 12:11:00.000890 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57320.2us
W20260709 12:11:00.009583 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 128622us
I20260709 12:10:59.977584 10332 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:00.014806 13473 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.014945 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.016219 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.017071 10583 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:00.018266 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:00.019593 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:00.019807 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.020406 10585 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:00.021786 10578 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:11:00.001596 13614 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:11:00.023840 13784 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.024533 13727 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.025336 13727 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:00.025913 13727 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:00.002316 13498 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:11:00.024495 13784 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.027191 10432 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:00.029042 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:00.035147 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 282451us
I20260709 12:11:00.036238 10438 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:11:00.035619 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:00.034631 10433 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:00.032249 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:00.045486 10431 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:00.045631 10437 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.051323 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:10:59.942598 13781 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:11:00.063745 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.064651 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.057075 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:00.050873 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:00.035341 13662 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.530s	user 0.016s	sys 0.001s
I20260709 12:11:00.032480 13653 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.075676 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.029218 13498 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:00.079267 10586 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:00.035097 13778 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:00.027100 13785 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:11:00.080888 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49873.4us
I20260709 12:11:00.028010 13660 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.026948 13727 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.098680 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.101445 10587 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:00.099946 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:00.101092 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:00.105175 10433 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:00.105757 10438 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:00.107290 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 65913.3us
I20260709 12:11:00.112275 10585 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:00.090037 13690 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.088579 13754 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.088868 13498 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:00.097473 13778 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.054004 13610 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:11:00.092514 13640 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:00.121019 10437 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:00.123431 10435 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:00.121332 13495 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:00.089062 13662 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:00.097074 13653 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.142783 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:00.115471 13793 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.133124 13495 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.131011 13639 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:11:00.137594 13795 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.133351 13710 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:11:00.133555 13764 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.139223 13799 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.139106 13697 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:11:00.144393 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:00.204613 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.125754 13778 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:00.122702 13690 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.142485 13640 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:00.126011 13754 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.165455 13785 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:00.219964 13796 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:11:00.223367 13498 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.151559 13683 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 12:11:00.239820 13792 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:11:00.241418 13785 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.241853 13763 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:00.250854 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 130808us
W20260709 12:11:00.350349 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 95420.6us
I20260709 12:11:00.248700 13498 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:00.166205 13793 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:00.361666 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.248919 13656 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.241472 13495 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.365274 10438 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:00.366976 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 258019us
W20260709 12:11:00.367816 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:00.373576 13809 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:11:00.373888 13662 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:11:00.231595 13764 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.390363 13710 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:00.390427 13441 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:00.391047 13710 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:00.391474 13441 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.392127 13710 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.393527 13710 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.394609 13441 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:00.369438 10553 leader_election.cc:336] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.661s (SENT)
I20260709 12:11:00.203220 13795 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:00.399528 13788 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.003s
I20260709 12:11:00.217269 13799 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.397960 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:00.343796 13787 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:11:00.366103 13780 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.375135 13807 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:11:00.398396 13697 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:00.443419 13697 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:00.444691 13697 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.445987 13614 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:00.446412 13614 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:00.447463 13614 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.448087 13791 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.449034 13614 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:00.449860 13763 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:00.450368 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:00.448763 13791 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.381695 13724 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.435581 13808 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.485204 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:00.480125 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:00.435896 13495 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:00.484455 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:00.506793 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 338958us
I20260709 12:11:00.513366 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.515450 10582 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:00.434741 13788 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.492169 13653 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:00.555833 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 186452us
I20260709 12:11:00.563858 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.509271 13593 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:00.441910 13640 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.440088 13780 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.496752 13495 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.618046 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:00.618099 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:00.580452 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.565083 13755 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:11:00.581095 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 147241us
I20260709 12:11:00.582547 10582 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:00.583607 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:00.588450 13799 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.590348 13784 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.598410 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:00.624377 10511 raft_consensus.cc:2410] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:11:00.622529 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.625985 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.575186 13781 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:11:00.663522 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45530.7us
I20260709 12:11:00.664563 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:00.664732 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.665776 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2065) took 1113 ms. Trace:
I20260709 12:11:00.672982 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:00.673866 10438 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:00.591673 13754 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.682137 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:00.676316 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:00.684743 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:00.585556 13690 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.511794 13808 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.687628 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:00.692839 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:00.693574 10582 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:00.701436 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 317535us
I20260709 12:11:00.703985 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:00.681327 13653 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:00.677647 13799 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.748497 10504 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:00.753662 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.748529 10431 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:00.756474 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.666298 10429 rpcz_store.cc:276] 0709 12:10:59.552737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.567860 (+ 15123us) service_pool.cc:224] Handling call
0709 12:10:59.600175 (+ 32315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:59.972746 (+372571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:59.972781 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:00.001760 (+ 28979us) log.cc:844] Serialized 52 byte log entry
0709 12:11:00.092951 (+ 91191us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:00.657265 (+564314us) log.cc:844] Serialized 14 byte log entry
0709 12:11:00.665176 (+  7911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.665347 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:00.665411 (+    64us) raft_consensus.cc:1685] finished
0709 12:11:00.665428 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:00.665669 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":844229,"prepare.queue_time_us":497704,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":390169,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":19607,"thread_start_us":136191,"threads_started":2,"wal-append.queue_time_us":173163}
I20260709 12:11:00.763842 10437 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:00.777845 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.778573 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:00.687669 13660 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:00.627914 13593 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:00.778267 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:00.814663 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 150185us
I20260709 12:11:00.820200 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:00.753371 13653 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.765272 10586 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:00.826086 10438 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:00.830137 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:00.779794 13767 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:11:00.835491 10512 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:11:00.753396 13781 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.820727 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:00.761163 13809 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:11:00.827284 13613 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.007s	sys 0.011s
W20260709 12:11:00.779275 13830 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:11:00.869462 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:00.859895 10504 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:11:00.861657 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:00.873514 10509 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:11:00.780527 13820 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:11:00.850666 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:00.834721 13653 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:00.887987 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:00.863471 13613 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.014s	sys 0.012s
I20260709 12:11:00.778203 13724 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:00.819171 13784 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.829488 13690 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:00.868603 10479 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:00.865808 13786 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.951s	user 0.020s	sys 0.005s
I20260709 12:11:00.789819 13793 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.896381 10584 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:00.890812 13791 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:00.930902 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:00.839164 13660 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.791020 13799 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:00.932412 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63961.4us
I20260709 12:11:00.933386 13827 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:00.934140 13816 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:11:00.966703 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:00.934167 13784 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.934864 13814 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:00.970603 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:00.918668 13656 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:00.937817 13498 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.572s	user 0.005s	sys 0.016s
I20260709 12:11:00.939045 13495 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.942250 13780 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.944244 13593 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.876857 13441 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:00.884269 13764 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:00.909749 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 59169us
W20260709 12:11:00.951058 13429 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.072s	user 0.014s	sys 0.002s
I20260709 12:11:00.951508 10586 raft_consensus.cc:2368] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6.
W20260709 12:11:00.919492 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.508s (SENT) [suppressed 23 similar messages]
W20260709 12:11:00.984486 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65040.9us
W20260709 12:11:00.985129 10553 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.518s (SENT)
W20260709 12:11:00.953931 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.987759 13784 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.987857 13814 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.988514 13784 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:00.990411 13814 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:00.984362 13660 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:00.991731 13832 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:11:00.981732 13788 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:00.994793 10580 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:11:00.974831 13816 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:00.995329 13816 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.955917 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:00.991640 11212 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:00.997076 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:00.998798 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:00.989017 13784 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:00.997045 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:01.001655 10553 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.599s (SENT)
I20260709 12:11:01.000116 13827 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:01.003536 10580 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:00.989611 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:01.009528 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:01.012909 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:01.014604 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:01.015161 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:01.015826 11221 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:00.887272 13781 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.014300 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:01.022636 13853 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:01.023309 13853 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:01.023772 13853 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.982400 13441 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.995718 13816 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.036334 11220 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:01.000133 13429 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:00.946734 13786 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:01.047607 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:01.049149 13786 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:01.051286 13822 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.999056 13824 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:01.051676 13429 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.035569 13441 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:01.060442 11221 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:01.070721 10582 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:01.069581 10581 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:11:01.083292 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:01.092837 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 75425.7us
I20260709 12:11:01.093472 10580 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:01.122968 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:01.124445 10478 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:01.124722 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:01.125439 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.053300 13660 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:01.095798 13798 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.025s	sys 0.060s
W20260709 12:11:01.129984 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 144151us
I20260709 12:11:01.064944 13781 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:01.071560 13791 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:01.157851 10581 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
W20260709 12:11:01.132289 10587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.005s	sys 0.018s
I20260709 12:11:01.063190 13822 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.132457 13660 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:01.179761 10587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.005s	sys 0.018s
I20260709 12:11:01.072319 13784 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:01.165984 13704 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.238s	user 0.011s	sys 0.007s
I20260709 12:11:01.186666 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:01.123425 13214 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:01.190802 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:01.192093 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 67711.9us
W20260709 12:11:01.193579 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 191972us
W20260709 12:11:01.169468 13798 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.028s	sys 0.062s
I20260709 12:11:01.163146 13819 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:01.132009 13429 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:11:01.253844 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57029.6us
W20260709 12:11:01.254397 10434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.007s	sys 0.032s
I20260709 12:11:01.259099 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:01.259334 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:01.189309 13710 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.795s	user 0.017s	sys 0.022s
W20260709 12:11:01.272845 10434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.008s	sys 0.033s
I20260709 12:11:01.128170 13824 consensus_queue.cc:1059] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 12:11:01.157740 13808 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.252771 13660 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.279910 13781 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:11:01.273711 10553 leader_election.cc:336] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.696s (SENT)
I20260709 12:11:01.280367 13781 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:01.280779 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:01.281428 13781 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:01.282439 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.260226 13814 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:11:01.270795 13704 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:01.283576 13799 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.275410 13851 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:01.284726 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.284531 13851 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.284981 13704 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:01.281005 13614 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.823s	user 0.017s	sys 0.020s
I20260709 12:11:01.286741 13791 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:01.287230 13710 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:01.287403 13791 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:01.287859 13710 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:01.289206 13441 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.290378 13710 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:01.291973 13851 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.292948 13822 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.293228 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:01.293823 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:01.274408 13429 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:01.295902 13429 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.297027 13795 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:01.297693 13795 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:01.298906 13660 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.299365 13791 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:01.191046 13755 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:01.300478 11221 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:01.303843 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2063) took 1077 ms. Trace:
I20260709 12:11:01.252737 13639 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:01.299621 10553 leader_election.cc:336] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.797s (SENT)
I20260709 12:11:01.324712 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:01.324946 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.304492 10587 rpcz_store.cc:276] 0709 12:11:00.226857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.227045 (+   188us) service_pool.cc:224] Handling call
0709 12:11:00.244183 (+ 17138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.244660 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.244709 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:00.256339 (+ 11630us) log.cc:844] Serialized 26 byte log entry
0709 12:11:01.249075 (+992736us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:01.249157 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.249274 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:01.303490 (+ 54216us) raft_consensus.cc:1685] finished
0709 12:11:01.303526 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:01.303747 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22449,"raft.queue_time_us":96,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":6459,"thread_start_us":873830,"threads_started":1,"wal-append.queue_time_us":910067}
I20260709 12:11:01.270457 13819 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:01.355899 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:01.357064 10511 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:01.295925 13441 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.362509 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:01.370108 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 340912us
I20260709 12:11:01.371558 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:01.322177 13639 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:01.165305 13827 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:01.355114 13572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.212s	user 0.000s	sys 0.033s
I20260709 12:11:01.369081 13791 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:01.356315 13819 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.402230 10511 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:11:01.425426 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 458) took 1065 ms. Trace:
I20260709 12:11:01.408828 13639 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:01.419466 13755 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:01.428570 13778 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.185s	user 0.016s	sys 0.000s
I20260709 12:11:01.428738 13754 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:01.419122 13773 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:01.428048 13572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.309s	user 0.000s	sys 0.034s
I20260709 12:11:01.460938 10586 rpcz_store.cc:276] 0709 12:11:00.360418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.985032 (+624614us) service_pool.cc:224] Handling call
0709 12:11:00.985435 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.985913 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.985942 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:00.986494 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:11:01.171503 (+185009us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:01.171545 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.171622 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:01.425140 (+253518us) raft_consensus.cc:1685] finished
0709 12:11:01.425162 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:01.425344 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1301,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":184586,"threads_started":1,"wal-append.queue_time_us":421186}
I20260709 12:11:01.565253 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.567181 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:01.568192 10511 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:01.530479 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:01.494120 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 211085us
I20260709 12:11:01.572168 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:01.580855 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:01.378021 13801 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.012s	sys 0.016s
I20260709 12:11:01.585503 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:01.400875 13814 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:01.597894 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.607774 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.608362 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:01.609143 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 81430.7us
I20260709 12:11:01.563153 13660 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:01.564754 13773 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:01.616308 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:01.584456 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.665s (SENT)
I20260709 12:11:01.611760 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.609712 10578 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:01.600087 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:01.617473 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:01.619755 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:01.620056 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:01.620450 10431 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:01.620656 10430 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
W20260709 12:11:01.587379 13801 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.014s	sys 0.017s
I20260709 12:11:01.622169 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:01.579221 13754 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:01.612526 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.006s	sys 0.022s
W20260709 12:11:01.635113 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.007s	sys 0.022s
I20260709 12:11:01.625298 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:01.585635 13804 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.373s	user 0.001s	sys 0.005s
I20260709 12:11:01.633457 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:01.636560 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:01.583490 13791 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:01.644018 13791 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:01.640615 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.641834 10587 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:01.647094 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.647223 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.648139 10578 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:11:01.642367 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:01.645506 10436 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.598701 13814 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:01.650345 10430 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:01.577216 13778 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.659582 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:01.660298 10587 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:01.661906 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:01.671819 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:01.672513 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:01.675552 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:01.652707 13660 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:01.681198 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:01.649699 13847 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:01.677914 10430 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:01.682765 10583 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:01.656251 13753 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:01.663704 13613 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:01.664959 13814 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:01.664319 13808 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:01.790156 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1740) took 1195 ms. Trace:
I20260709 12:11:01.663942 13838 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:11:01.694559 13613 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:01.851307 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 214887us
I20260709 12:11:01.693352 13847 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:01.803263 13753 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:01.796491 10508 rpcz_store.cc:276] 0709 12:11:00.594556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.594754 (+   198us) service_pool.cc:224] Handling call
0709 12:11:00.675619 (+ 80865us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.781150 (+105531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.781187 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:01.089066 (+307879us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b2c0001c5e8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:01.114461 (+ 25395us) log.cc:844] Serialized 26 byte log entry
0709 12:11:01.357693 (+243232us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:01.357734 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.357823 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:01.789596 (+431773us) raft_consensus.cc:1685] finished
0709 12:11:01.789633 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:01.790017 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140528,"raft.queue_time_us":106838,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":28915,"spinlock_wait_cycles":24193280,"thread_start_us":197053,"threads_started":1,"wal-append.queue_time_us":487331}
I20260709 12:11:02.070941 10512 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:01.868687 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 267790us
I20260709 12:11:02.072434 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.066231 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.074779 10432 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:02.075778 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.076498 10431 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:11:02.090289 10430 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:02.091229 10513 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:02.080909 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.047873 10587 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:01.853207 13613 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:02.079310 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 494859us
W20260709 12:11:01.973618 13859 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:11:02.102465 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:01.892541 13863 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 12:11:02.104714 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:01.826444 13808 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.119612 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.119472 10436 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:11:02.112236 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:02.103753 13429 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.807s	user 0.013s	sys 0.000s
I20260709 12:11:01.971516 13856 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:01.973789 13862 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:01.972162 13848 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:01.800891 13660 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:02.039111 13822 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.120013 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.075302 13857 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:02.126317 13849 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.299s	user 0.001s	sys 0.001s
I20260709 12:11:02.146273 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.161041 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.161706 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.136765 13856 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:02.132507 13429 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.184124 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:02.185184 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52646.2us
I20260709 12:11:02.141563 13791 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:02.189827 10431 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:02.130905 13258 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 12:11:02.191481 10553 leader_election.cc:336] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.529s (SENT)
I20260709 12:11:02.193593 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:02.194943 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.158950 13778 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:02.155949 13868 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:02.203724 13847 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:02.205026 13858 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
W20260709 12:11:02.207887 13874 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:11:02.207494 13660 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:02.151854 13822 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.210219 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.210901 10581 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:02.212065 13822 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.151582 13857 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.183378 13856 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:02.165596 13827 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:11:02.238549 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.258076 13753 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:02.214883 13865 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.275s	user 0.000s	sys 0.000s
I20260709 12:11:02.259030 13753 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:02.259783 13827 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:02.259723 13753 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.260334 13827 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:02.184994 13799 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:02.151291 13848 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.187685 13862 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.277402 13799 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:02.235117 13808 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.392233 13799 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.216773 13813 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:02.393308 13813 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:02.394351 13813 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:02.360215 13853 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.336s	user 0.013s	sys 0.004s
I20260709 12:11:02.219728 13880 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.396302 13853 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.396678 13880 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.397802 10582 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:02.399420 13880 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.199579 13791 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:02.288321 10431 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:02.400556 13813 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.227244 13868 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:02.403677 13868 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:02.305907 13653 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.390s	user 0.012s	sys 0.005s
I20260709 12:11:02.388124 13857 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:02.370829 13816 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.280s	user 0.016s	sys 0.000s
I20260709 12:11:02.391371 13862 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.406730 13653 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.410269 13653 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:02.410955 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.412200 10510 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:02.412460 13816 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.413776 10507 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:02.414165 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1741) took 1819 ms. Trace:
I20260709 12:11:02.415005 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2123) took 1427 ms. Trace:
I20260709 12:11:02.415530 10507 rpcz_store.cc:276] 0709 12:11:00.987680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.013934 (+ 26254us) service_pool.cc:224] Handling call
0709 12:11:02.414727 (+1400793us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400198cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:02.414902 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3204305792}
I20260709 12:11:02.412212 13862 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:02.420125 10332 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 6, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 6 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:02.421666 13781 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.918038  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:02.422966 13816 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.426919 13754 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:02.426781 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:02.432466 13803 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 2.228s	user 0.000s	sys 0.003s
W20260709 12:11:02.433723 13823 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.652s	user 0.002s	sys 0.002s
I20260709 12:11:02.435710 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2055) took 2396 ms. Trace:
I20260709 12:11:02.436111 10579 rpcz_store.cc:276] 0709 12:11:00.039358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.039643 (+   285us) service_pool.cc:224] Handling call
0709 12:11:00.063132 (+ 23489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.104185 (+ 41053us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:11:00.104223 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:00.133657 (+ 29434us) log.cc:844] Serialized 26 byte log entry
0709 12:11:00.558394 (+424737us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:00.558445 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.558527 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:02.433086 (+1874559us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:02.435338 (+  2252us) raft_consensus.cc:1685] finished
0709 12:11:02.435364 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:02.435616 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69546,"raft.queue_time_us":67437,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9189,"spinlock_wait_cycles":54333824,"thread_start_us":356461,"threads_started":1,"wal-append.queue_time_us":484610}
W20260709 12:11:02.440742 10583 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2056) took 2401 ms (client timeout 1503 ms). Trace:
W20260709 12:11:02.441686 10335 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 221) took 29461 ms (client timeout 15000 ms). Trace:
W20260709 12:11:02.453593 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 1.1824e+06us
I20260709 12:11:02.455545 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.452790 13826 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 12:11:02.458557 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.458868 13798 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:02.459486 13798 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:02.460017 13818 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:02.460510 13798 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.460664 13818 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:02.464210 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.465378 13498 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.465487 13593 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:02.466001 13498 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.466109 13593 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.454692 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2166) took 1139 ms. Trace:
I20260709 12:11:02.466854 13811 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:02.467707 13826 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:02.467478 13811 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.468146 13826 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:02.470060 13811 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:02.469727 13826 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.448561 13773 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.471220 13593 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.472421 13826 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.414593 10510 rpcz_store.cc:276] 0709 12:11:00.595154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.595508 (+   354us) service_pool.cc:224] Handling call
0709 12:11:02.413924 (+1818416us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b540036db54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:02.414084 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3973086464}
I20260709 12:11:02.438761 10579 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:11:02.473434 10579 pending_rounds.cc:85] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 4
I20260709 12:11:02.473701 10579 pending_rounds.cc:107] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:11:02.473933 10579 raft_consensus.cc:2889] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:02.467197 10582 rpcz_store.cc:276] 0709 12:11:01.315732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.316004 (+   272us) service_pool.cc:224] Handling call
0709 12:11:01.318448 (+  2444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:02.398216 (+1079768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:02.454379 (+ 56163us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540010aed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:02.454614 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2481797888}
I20260709 12:11:02.476063 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.478170 13818 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.478266 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.479254 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:02.422003 10332 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:11:02.441359 10583 rpcz_store.cc:269] 0709 12:11:00.039711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.039868 (+   157us) service_pool.cc:224] Handling call
0709 12:11:02.440403 (+2400535us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b540010b8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:02.440610 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3711994240}
I20260709 12:11:02.481098 13214 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:02.481544 10332 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 5, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:02.482657 13498 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:02.483561 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.471992 13773 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.485015 10332 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:11:02.485647 10437 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2090) took 1864 ms (client timeout 1540 ms). Trace:
W20260709 12:11:02.486050 10437 rpcz_store.cc:269] 0709 12:11:00.621195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.621348 (+   153us) service_pool.cc:224] Handling call
0709 12:11:02.485428 (+1864080us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b54002661d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:02.485542 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180453120}
I20260709 12:11:02.481578 13214 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:02.490908 13697 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:02.500731 13864 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.638s	user 0.002s	sys 0.000s
I20260709 12:11:02.502914 10432 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:02.505178 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2085) took 1920 ms. Trace:
I20260709 12:11:02.506565 10332 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 6, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 6 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:02.505654 10434 rpcz_store.cc:276] 0709 12:11:00.584958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.585710 (+   752us) service_pool.cc:224] Handling call
0709 12:11:00.586198 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:01.293403 (+707205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:01.293445 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:01.294170 (+   725us) log.cc:844] Serialized 52 byte log entry
0709 12:11:01.427865 (+133695us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:01.427925 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.428049 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:02.499914 (+1071865us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:02.504786 (+  4872us) raft_consensus.cc:1685] finished
0709 12:11:02.504824 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:02.505110 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1274,"raft.queue_time_us":43452,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":18543,"spinlock_wait_cycles":20551040,"thread_start_us":787054,"threads_started":2,"wal-append.queue_time_us":349113}
W20260709 12:11:02.513535 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 322074us
W20260709 12:11:02.514719 10553 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.758s (SENT)
I20260709 12:11:02.515260 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.515393 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.516390 10512 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:02.516942 13795 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:11:02.520102 13875 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 12:11:02.521307 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.525035 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1753) took 1653 ms. Trace:
I20260709 12:11:02.525326 13868 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.525514 10504 rpcz_store.cc:276] 0709 12:11:00.871600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.523981 (+1652381us) service_pool.cc:224] Handling call
0709 12:11:02.524430 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:02.524663 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:02.524688 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:02.524736 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:02.524770 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:02.524835 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:02.524963 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:02.450606 13613 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.528254 10578 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:02.528312 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:02.531909 13877 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:11:02.532337 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:02.533313 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.532459 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.536749 13795 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:02.537439 13795 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.539487 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:02.540148 10512 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:02.541165 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:02.541846 10510 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:02.541178 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.544623 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:02.515192 13810 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:02.545933 13810 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.549028 13810 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.549623 13810 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:02.550148 13848 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:02.550089 13810 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.551450 13429 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.552052 13795 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:02.552126 13429 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:02.552578 10510 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:02.552897 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.554231 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.554399 13786 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:02.556522 13690 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:02.557430 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.558084 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.558390 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.560185 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.562875 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.590221 10582 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:02.564397 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.580719 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.591501 10578 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:02.582597 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:02.591655 10553 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.892s (SENT)
I20260709 12:11:02.589577 10438 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:02.592417 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.595554 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.617865 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.618674 10586 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.618769 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.619946 10582 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:02.619869 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 61565.5us
W20260709 12:11:02.580001 13819 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.050s	user 0.005s	sys 0.011s
I20260709 12:11:02.621263 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:02.595840 10332 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 6, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 6 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } }
I20260709 12:11:02.617713 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.620261 10581 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:02.640273 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.640942 10581 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:02.622073 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.642516 10432 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
W20260709 12:11:02.580583 13639 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.002s	user 0.009s	sys 0.005s
I20260709 12:11:02.581198 13613 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:02.644543 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45069.5us
I20260709 12:11:02.655674 13868 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.656597 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:02.657300 11213 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:02.654791 13833 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:11:02.659937 13697 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:02.660262 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:02.661345 13810 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.661593 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:02.662493 13822 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:02.662593 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:02.663427 13822 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:02.563508 13853 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:02.630889 13840 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.664662 10505 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:02.645807 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.631700 13795 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:11:02.664997 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:02.629827 13498 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.666309 13498 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.667084 13763 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:11:02.454185 10335 rpcz_store.cc:269] 0709 12:10:32.979797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.980071 (+   274us) service_pool.cc:224] Handling call
0709 12:10:34.471989 (+1491918us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5400270010. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb0b2 00007f2c98934759 00007f2c988cca76 00007f2c9889dd58 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc
0709 12:11:02.441398 (+27969409us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:52.988876 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:52.989548 (+   672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:52.989587 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:10:52.989605 (+    18us) tablet.cc:662] Decoding operations
0709 12:10:52.990188 (+   583us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:52.990258 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:52.990285 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:10:52.990615 (+   330us) tablet.cc:701] Row locks acquired
0709 12:10:52.990633 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:10:52.990791 (+   158us) write_op.cc:270] Start()
0709 12:10:52.990905 (+   114us) write_op.cc:276] Timestamp: P: 1783599052990739 usec, L: 0
0709 12:10:52.990928 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:10:53.060299 (+ 69371us) log.cc:844] Serialized 1294 byte log entry
0709 12:10:53.842312 (+782013us) op_driver.cc:464] REPLICATION: finished
0709 12:10:54.144507 (+302195us) write_op.cc:301] APPLY: starting
0709 12:10:54.150269 (+  5762us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:54.213535 (+ 63266us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:54.237522 (+ 23987us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:54.343180 (+105658us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:54.381820 (+ 38640us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:10:54.381870 (+    50us) write_op.cc:312] APPLY: finished
0709 12:10:54.382099 (+   229us) log.cc:844] Serialized 35 byte log entry
0709 12:10:54.382288 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:54.382398 (+   110us) write_op.cc:454] Released schema lock
0709 12:10:54.382439 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:54.382889 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:54.382918 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":801556,"reported_tablets":44,"spinlock_wait_cycles":3008384,"child_traces":[["op",{"apply.queue_time_us":270458,"apply.run_cpu_time_us":3780,"apply.run_wall_time_us":249958,"num_ops":3,"prepare.queue_time_us":355165,"prepare.run_cpu_time_us":14160,"prepare.run_wall_time_us":189248,"raft.queue_time_us":386391,"raft.run_cpu_time_us":62159,"raft.run_wall_time_us":290245,"replication_time_us":812947,"thread_start_us":889676,"threads_started":4,"wal-append.queue_time_us":129608,"wal-append.run_cpu_time_us":18638,"wal-append.run_wall_time_us":2189882}]]}
I20260709 12:11:02.668691 13498 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:02.669703 13697 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:02.669857 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.670370 13697 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:02.648685 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:02.649901 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.628949 13786 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.673552 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.649633 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.676987 10510 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:02.677436 10504 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:02.679658 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.681079 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:02.681135 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.686833 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.689850 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.690024 13787 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:02.645159 13653 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:11:02.689889 13690 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.623353 13429 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.696753 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:02.690909 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.675837 10332 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:11:02.703634 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 112017us
I20260709 12:11:02.717722 10586 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
W20260709 12:11:02.746114 10553 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.950s (SENT)
I20260709 12:11:02.746538 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.747255 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.719336 10512 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:02.748502 10511 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:02.722093 13814 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.918s	user 0.015s	sys 0.000s
I20260709 12:11:02.497152 13214 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.697815 13786 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:02.727092 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45977.6us
I20260709 12:11:02.778788 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.698417 13813 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:02.647217 13258 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:02.740872 10578 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:02.741888 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.720595 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:02.751411 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.792570 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:02.793490 10578 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:02.674938 13639 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.753403 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.754112 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.754489 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.674431 13816 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:02.701221 13781 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:02.777856 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:02.777100 13856 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.001s
I20260709 12:11:02.721546 13787 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:02.732065 10581 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:02.786245 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.786628 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.789250 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.790833 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.790627 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.798121 10430 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:02.722819 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:02.664680 13840 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.804283 10586 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:11:02.671468 13697 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:02.845461 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 99405.6us
I20260709 12:11:02.804571 10504 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:02.748322 13778 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:02.786784 10332 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 3, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:02.806156 10582 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:02.807324 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.830821 10508 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:02.836170 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.835062 10433 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:02.837070 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114357us
I20260709 12:11:02.648185 13819 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.779361 13753 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:02.844936 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 54395us
I20260709 12:11:02.866331 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.862172 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:02.845815 10505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1748) took 2097 ms (client timeout 2175 ms). Trace:
I20260709 12:11:02.851511 10429 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:02.872128 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.846225 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.873054 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.850385 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:02.861478 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.864826 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.867198 10436 raft_consensus.cc:2368] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
I20260709 12:11:02.865873 10507 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:02.874897 10432 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:11:02.856138 10553 leader_election.cc:336] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.095s (SENT)
I20260709 12:11:02.876528 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.868220 10511 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:11:02.838572 13851 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:02.865527 13857 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:02.879236 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.879361 13710 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:02.879652 10332 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:02.881080 13710 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:02.881335 10332 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:02.804494 13862 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:02.879281 13851 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.839804 13816 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:02.885125 13857 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:02.886232 13857 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.887432 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.889492 13824 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:02.844775 13786 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:02.890165 13824 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:02.890522 13786 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:02.891041 13786 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:02.891359 13824 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.893540 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.895242 13710 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:02.895767 13710 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:02.896194 13710 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.894773 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.900542 13639 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.900938 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:02.901180 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45104.3us
I20260709 12:11:02.901631 13791 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:02.901937 10504 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:11:02.902052 10553 leader_election.cc:336] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.110s (SENT)
I20260709 12:11:02.902540 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.903815 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:02.904018 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:02.904621 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.803519 13763 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:11:02.871707 10505 rpcz_store.cc:269] 0709 12:11:00.747959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.753080 (+  5121us) service_pool.cc:224] Handling call
0709 12:11:02.845494 (+2092414us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400229cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:02.845724 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4242179968}
I20260709 12:11:02.907970 11220 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:11:02.908511 11220 pending_rounds.cc:85] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 7
I20260709 12:11:02.908847 11220 pending_rounds.cc:107] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:11:02.908731 10512 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:02.909379 11220 raft_consensus.cc:2889] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:02.908509 10508 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:11:02.841432 13787 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:02.912868 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:02.915772 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.904490 10578 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:02.916847 10513 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:02.904652 10430 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:11:02.779503 13814 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.835978 13822 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:02.803860 13781 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:02.905635 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.919220 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:02.920858 10429 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:02.930404 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.923147 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.861884 13778 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:02.841046 13856 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:02.901281 13639 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:02.910250 13851 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:02.925014 10332 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:02.926010 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:02.932476 10580 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.014s	sys 0.010s
W20260709 12:11:02.949402 10580 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.014s	sys 0.010s
I20260709 12:11:02.952565 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.953349 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.954344 10434 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:02.960983 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58956.1us
W20260709 12:11:02.962452 10553 leader_election.cc:336] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.181s (SENT)
I20260709 12:11:02.963068 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:02.932422 13868 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:02.938175 13801 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:02.974802 10587 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:02.981520 10332 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 4 to 6, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 6 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:02.973877 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.968034 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.965634 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.977015 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.967831 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.978349 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.983996 10433 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:02.984331 10436 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:02.978932 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.985618 10432 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:11:02.977586 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.940554 13778 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:02.981680 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.921324 13813 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:11:02.986908 10438 raft_consensus.cc:2410] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 12:11:02.990594 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:02.955401 13656 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:02.973572 13763 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:02.992012 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.992280 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.997663 10431 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:11:02.994854 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.998294 10430 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:02.999179 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.995800 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:02.992070 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:02.999346 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:02.993191 10332 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 4 to 6, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 6 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:02.995671 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:03.023768 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:03.012351 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.007068 10578 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:03.028407 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:03.028553 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.027092 10432 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:03.029649 10433 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:11:03.010746 10429 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:03.026434 10437 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:03.031229 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.012s	sys 0.015s
I20260709 12:11:02.965392 13853 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:02.944469 13822 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.039326 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:03.035386 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:03.035455 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.012s	sys 0.015s
I20260709 12:11:03.032433 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.037139 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.043349 10438 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:02.994742 13826 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.522s	user 0.002s	sys 0.011s
W20260709 12:11:03.039284 11215 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.013s	sys 0.016s
I20260709 12:11:03.048792 10432 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:03.055685 10432 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:03.057955 11215 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.013s	sys 0.016s
W20260709 12:11:03.049104 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.009s	sys 0.022s
W20260709 12:11:02.994992 13827 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.734s	user 0.010s	sys 0.003s
I20260709 12:11:02.988884 13639 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:03.024736 13794 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.004s	sys 0.023s
I20260709 12:11:03.009060 13214 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:03.061021 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.032403 10332 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 2 to 4, term changed from 6 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to <none>, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67), VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 7 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:03.011380 13829 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:03.056555 10432 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.069599 10429 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:03.073275 10429 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:03.074177 10432 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:03.035241 13801 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:03.036321 13811 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:03.061340 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.009s	sys 0.022s
I20260709 12:11:03.071702 10430 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:03.039618 13791 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:03.041087 13868 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:03.084769 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:03.107882 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.051385 13790 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:03.109205 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.101738 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:03.114925 10435 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:03.118218 11212 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:03.090868 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.123839 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:03.069142 13794 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.004s	sys 0.024s
I20260709 12:11:03.109025 10429 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.122659 10431 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
W20260709 12:11:03.139351 10506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.004s	sys 0.024s
W20260709 12:11:03.139905 10506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.005s	sys 0.025s
I20260709 12:11:03.142020 10332 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 7, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:03.144299 11213 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:03.131536 13811 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.144488 10332 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:03.148144 13814 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.149084 10429 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:03.076995 13827 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.150898 13639 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.049548 13853 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.153225 13827 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.154206 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.115144 13801 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:03.155663 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:03.129452 10434 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:03.156597 10578 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:03.157609 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.158396 10436 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:03.158679 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.159250 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.159543 10433 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:03.159806 11216 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:03.160432 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.047111 13656 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:03.161062 10509 raft_consensus.cc:2368] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6.
I20260709 12:11:03.031522 13763 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.064209 13613 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:03.076203 13798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.020s	sys 0.014s
I20260709 12:11:03.168864 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.187887 10431 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:11:03.123152 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.167450 13853 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:03.180269 13801 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.109264 13429 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:03.148056 13811 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.199157 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.200219 10435 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:03.202013 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.202962 10436 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:03.162600 13826 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.165223 13640 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:03.177238 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.032194 13781 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:03.184636 13798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.020s	sys 0.018s
I20260709 12:11:03.217680 13840 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:03.230923 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.232349 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:03.232671 13763 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:03.232972 10581 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:03.234431 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:03.235064 10586 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:03.234653 13868 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:03.218567 13834 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 2.341s	user 0.000s	sys 0.008s
I20260709 12:11:03.222301 13816 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:03.224079 13822 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:03.225522 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.226505 10512 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:03.254206 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.255494 10513 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:03.261063 10508 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:03.268921 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.248255 10586 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:03.270658 10436 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:03.192047 10429 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:03.249931 10578 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:03.252126 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2062) took 3058 ms. Trace:
I20260709 12:11:03.275041 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.248538 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.239099 13856 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:03.226934 13819 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.280049 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.281196 10430 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:03.283675 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.287894 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.297672 10578 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:11:03.298393 10586 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:03.274431 10585 rpcz_store.cc:276] 0709 12:11:00.193871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.194147 (+   276us) service_pool.cc:224] Handling call
0709 12:11:00.207556 (+ 13409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.440465 (+232909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.440502 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:00.441813 (+  1311us) log.cc:844] Serialized 26 byte log entry
0709 12:11:00.671720 (+229907us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:00.671797 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.072034 (+400237us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54003d0254. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:01.072088 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:02.665552 (+1593464us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:03.218161 (+552609us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b24001fb4e8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa3109d 00007f2c9266310e 00007f2c9267a88c 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221
0709 12:11:03.251811 (+ 33650us) raft_consensus.cc:1685] finished
0709 12:11:03.251839 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:03.252026 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79581,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":42972,"spinlock_wait_cycles":575097344,"thread_start_us":153339,"threads_started":1,"wal-append.queue_time_us":275123}
I20260709 12:11:03.286197 13786 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.307679 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:03.308676 10586 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:11:03.310129 10581 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:03.315611 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:03.312006 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 366731us
I20260709 12:11:03.289937 13781 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:03.322126 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:03.323149 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.329156 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.330443 10585 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:03.225793 13826 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.354930 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.356202 10578 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:11:03.359938 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.361408 10585 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:03.313812 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2095) took 1127 ms. Trace:
I20260709 12:11:03.369755 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:03.374919 10581 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:03.384783 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1744) took 2721 ms. Trace:
I20260709 12:11:03.353605 13819 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:03.392366 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.387823 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:03.366657 10580 rpcz_store.cc:276] 0709 12:11:02.186770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.429010 (+242240us) service_pool.cc:224] Handling call
0709 12:11:02.429293 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:02.429534 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:02.950942 (+521408us) log.cc:844] Serialized 14 byte log entry
0709 12:11:03.312954 (+362012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:03.313090 (+   136us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:03.313208 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:03.313383 (+   175us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:03.313676 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6392,"prepare.queue_time_us":683590,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":6676,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":5398,"thread_start_us":875176,"threads_started":2}
I20260709 12:11:03.317586 13778 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.273237 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:03.493180 10508 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:03.493875 10508 raft_consensus.cc:614] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:03.387552 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.484789 10580 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:03.511168 10586 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:03.512741 10586 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:03.516486 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 471314us
I20260709 12:11:03.517088 10586 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:11:03.520251 10586 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:03.385396 10506 rpcz_store.cc:276] 0709 12:11:00.651480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.681663 (+ 30183us) service_pool.cc:224] Handling call
0709 12:11:00.751037 (+ 69374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.814392 (+ 63355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.814426 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:00.838554 (+ 24128us) log.cc:844] Serialized 26 byte log entry
0709 12:11:02.462140 (+1623586us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b180033e380. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:03.140396 (+678256us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:03.140454 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:03.140570 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:03.368738 (+228168us) raft_consensus.cc:1685] finished
0709 12:11:03.368770 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:03.372907 (+  4137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75192,"raft.queue_time_us":66845,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":10037,"spinlock_wait_cycles":21395328,"thread_start_us":677975,"threads_started":1,"wal-append.queue_time_us":2342032}
W20260709 12:11:03.522922 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:03.524119 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.315554 13656 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.506366 13816 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.549865 10579 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:03.555950 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:03.532433 10553 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.765s (SENT)
I20260709 12:11:03.543040 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.556025 10581 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:03.507052 13825 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:03.562207 10510 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:03.522118 13819 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:03.550314 13856 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.573055 10585 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:03.574321 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.576311 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.578125 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.580802 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:03.579129 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46771.5us
I20260709 12:11:03.584748 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:03.586751 10553 leader_election.cc:336] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.904s (SENT)
I20260709 12:11:03.587364 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:03.595570 10553 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.961s (SENT)
I20260709 12:11:03.596146 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:03.599126 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:03.588454 13891 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:11:03.591747 13498 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.626009 13906 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:03.630334 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:03.611636 13785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.013s	sys 0.031s
I20260709 12:11:03.567039 13816 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:03.639786 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.587970 13856 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.635478 13498 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:03.652074 10509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1754) took 2753 ms (client timeout 1581 ms). Trace:
W20260709 12:11:03.652666 10509 rpcz_store.cc:269] 0709 12:11:00.903722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.478910 (+1575188us) service_pool.cc:224] Handling call
0709 12:11:03.651765 (+1172855us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b5400360454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:03.651972 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566242048}
W20260709 12:11:03.636747 13785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.013s	sys 0.031s
I20260709 12:11:03.658231 13856 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:03.658764 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.659684 10509 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:03.659516 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.660497 10506 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:03.661638 13498 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:03.663653 13888 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.310s	user 0.001s	sys 0.001s
W20260709 12:11:03.664656 13893 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.345s	user 0.002s	sys 0.000s
I20260709 12:11:03.665576 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 473) took 2373 ms. Trace:
I20260709 12:11:03.666188 11215 rpcz_store.cc:276] 0709 12:11:01.291964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.294426 (+  2462us) service_pool.cc:224] Handling call
0709 12:11:01.294827 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:02.456901 (+1162074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:02.456939 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:02.457543 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:11:03.059063 (+601520us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:03.059108 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:03.059184 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:03.665204 (+606020us) raft_consensus.cc:1685] finished
0709 12:11:03.665237 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:03.665462 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1161560,"raft.queue_time_us":1164260,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":601099,"threads_started":1,"wal-append.queue_time_us":835832}
W20260709 12:11:03.669919 13690 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.925s	user 0.006s	sys 0.009s
I20260709 12:11:03.670468 13690 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:03.665732 10580 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:03.672752 13885 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.673491 13885 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.675220 13886 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:03.675993 13886 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.678315 13826 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.686707 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.688591 10586 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:03.689921 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.692281 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.693060 10435 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:03.678645 13886 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.717101 10586 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:03.704581 13816 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.679848 13819 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.707871 13885 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:03.709627 13897 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:11:03.711241 13778 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:03.713752 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:03.691959 13856 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.730065 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.752321 10586 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
W20260709 12:11:03.716565 13787 consensus_meta.cc:287] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.005s
I20260709 12:11:03.757253 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2107) took 1189 ms. Trace:
I20260709 12:11:03.762739 10508 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:03.721606 13886 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:03.764297 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 90567.2us
I20260709 12:11:03.767426 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:03.767568 10578 rpcz_store.cc:276] 0709 12:11:02.567905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:03.516249 (+948344us) service_pool.cc:224] Handling call
0709 12:11:03.560402 (+ 44153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:03.560669 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:03.571009 (+ 10340us) log.cc:844] Serialized 14 byte log entry
0709 12:11:03.756850 (+185841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:03.756911 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:03.756962 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:03.757046 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:03.757182 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18721,"prepare.queue_time_us":35416,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":525,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3517,"thread_start_us":184923,"threads_started":1}
I20260709 12:11:03.747685 13885 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:03.775681 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.725528 13724 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:11:03.730717 13857 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:11:03.780908 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:03.788918 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:03.760145 13763 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:03.760267 13791 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:11:03.796022 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2100) took 1233 ms. Trace:
I20260709 12:11:03.801151 10584 rpcz_store.cc:276] 0709 12:11:02.562193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.564922 (+  2729us) service_pool.cc:224] Handling call
0709 12:11:02.630047 (+ 65125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:02.701403 (+ 71356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:02.701441 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:02.702073 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 12:11:03.052316 (+350243us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:03.052386 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:03.052499 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:03.795698 (+743199us) raft_consensus.cc:1685] finished
0709 12:11:03.795732 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:03.795934 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63018,"raft.queue_time_us":64557,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"thread_start_us":348809,"threads_started":1,"wal-append.queue_time_us":837463}
I20260709 12:11:03.799805 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.803874 10508 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:03.760771 13787 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.781334 13906 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:03.757063 13856 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:03.800618 13710 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.904s	user 0.012s	sys 0.006s
I20260709 12:11:03.762465 13781 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:11:03.817888 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:03.819015 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:03.812325 10403 connection.cc:256] RPC call timeout handler was delayed by -1.0870322989999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 12:11:03.821125 10403 connection.cc:256] RPC call timeout handler was delayed by -1.0870322989999295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:11:03.788523 13886 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:03.821460 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:03.821563 10403 connection.cc:256] RPC call timeout handler was delayed by -1.0798322989999178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 12:11:03.824411 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:03.826154 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:03.826275 13906 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:03.826706 13906 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:03.826936 10403 connection.cc:256] RPC call timeout handler was delayed by -1.0690322989999004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 12:11:03.826803 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:03.827196 13906 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:03.827962 10403 leader_election.cc:400] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:03.827965 13853 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:03.828393 10403 leader_election.cc:403] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:11:03.831095 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:03.832382 13853 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:03.791226 13885 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.836472 11177 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:03.840202 13826 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:03.840651 13826 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:03.841046 13826 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:03.841401 13885 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.842396 13885 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.817044 13778 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:03.847973 13924 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:03.817545 13710 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.848758 13924 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:03.851343 10505 raft_consensus.cc:2368] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:03.847759 13927 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:03.842943 13885 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.856805 13885 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.857995 13710 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.865152 13785 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:03.873219 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.880587 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.880343 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:03.865804 13785 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:03.887717 13640 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:03.888024 13906 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.846717 13856 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:03.915452 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.903533 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.903700 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.905006 10578 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:03.905290 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.905695 10506 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:03.907191 13798 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:03.907755 13886 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.908353 10438 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:03.924572 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.907796 13787 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.908676 13785 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:03.925508 10435 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:03.926179 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.926376 13785 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:03.926606 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.929034 13826 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:03.912874 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.914264 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:03.915441 13901 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.301s	user 0.005s	sys 0.000s
I20260709 12:11:03.915552 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.902139 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:03.935850 10584 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:03.921380 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.937168 10430 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:03.923774 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:03.938016 10511 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:03.914599 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:03.939613 10431 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:03.905562 13640 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:03.941573 10508 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:03.920501 13798 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:03.962872 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.963287 13798 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:03.963860 10432 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:03.945201 11215 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:03.967049 10432 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:11:03.957231 13931 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.003s
I20260709 12:11:03.957785 13787 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:03.958636 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.969076 10434 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:03.940487 13924 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.966944 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.972492 10436 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:03.954208 13908 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.329s	user 0.000s	sys 0.002s
I20260709 12:11:03.973904 10430 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:03.975942 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:03.976877 10430 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:03.979719 10430 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:03.985558 10430 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:03.986862 10434 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
W20260709 12:11:03.987843 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 159526us
I20260709 12:11:03.988279 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.988368 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:03.989281 10432 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:11:03.989419 10434 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:03.992439 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:03.996798 13826 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:03.999449 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2102) took 1434 ms. Trace:
I20260709 12:11:03.999871 10582 rpcz_store.cc:276] 0709 12:11:02.564821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.645492 (+ 80671us) service_pool.cc:224] Handling call
0709 12:11:03.999222 (+1353730us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540017a054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:03.999379 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302854400}
I20260709 12:11:04.007965 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:04.008805 10437 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:04.017031 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:04.021795 11216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 491) took 1385 ms. Trace:
W20260709 12:11:04.023337 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 192344us
I20260709 12:11:04.023747 10434 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:04.023483 11216 rpcz_store.cc:276] 0709 12:11:02.625866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.626002 (+   136us) service_pool.cc:224] Handling call
0709 12:11:03.159580 (+533578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:03.160197 (+   617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.010733 (+850536us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b5400830750. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:04.010959 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143609088}
I20260709 12:11:04.029009 10436 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:04.032642 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44024.3us
I20260709 12:11:04.033692 10437 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:04.028803 10430 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:04.037911 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 157729us
I20260709 12:11:03.992471 13441 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:04.042003 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:04.043602 10432 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:04.045261 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.049924 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:04.052277 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.053094 10430 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:04.050182 13787 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:04.050645 13793 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:04.031329 13785 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:04.031112 13794 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:04.058028 10438 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:04.051514 13870 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:04.059751 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:04.080196 10438 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:04.082592 10430 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:04.053231 13710 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:11:04.058125 13787 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:04.074390 13819 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:04.087692 10438 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.080116 13794 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:04.083655 13885 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:04.047077 13441 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.106643 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.091964 13787 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.091086 13924 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:04.112915 10579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2181) took 1436 ms (client timeout 1842 ms). Trace:
I20260709 12:11:04.100008 13639 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:04.117412 13786 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.132025 13639 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:04.132763 13639 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.141942 13943 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:04.098928 13498 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:11:04.112874 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:04.089375 13785 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.110193 13794 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:04.128765 10579 rpcz_store.cc:269] 0709 12:11:02.676638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.690583 (+ 13945us) service_pool.cc:224] Handling call
0709 12:11:04.112619 (+1422036us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400090954. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:04.112826 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1841053568}
I20260709 12:11:04.102336 13819 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:04.081579 13933 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 12:11:04.152501 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 98707.2us
I20260709 12:11:04.157094 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:04.156988 10587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2087) took 2545 ms (client timeout 3176 ms). Trace:
W20260709 12:11:04.160097 10587 rpcz_store.cc:269] 0709 12:11:01.611503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.104254 (+492751us) service_pool.cc:224] Handling call
0709 12:11:04.156713 (+2052459us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b5400178754. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:04.156899 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978261888}
I20260709 12:11:04.100143 13927 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:11:04.101963 13929 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
W20260709 12:11:04.179322 10553 leader_election.cc:336] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.539s (SENT)
I20260709 12:11:04.180073 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.181816 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:04.153934 13786 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:04.182724 10506 raft_consensus.cc:606] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.156512 13939 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:04.153448 13943 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.192252 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:04.193589 10587 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:04.195910 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:04.204901 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 116949us
W20260709 12:11:04.197114 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67375.1us
I20260709 12:11:04.206795 10579 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:04.207618 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.208196 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.208837 10582 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:04.214927 13498 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:04.215572 13498 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:04.216039 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.216665 13498 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:04.218035 10507 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2130) took 1566 ms (client timeout 1541 ms). Trace:
I20260709 12:11:04.220307 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:04.218729 10507 rpcz_store.cc:269] 0709 12:11:02.651811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.669498 (+ 17687us) service_pool.cc:224] Handling call
0709 12:11:04.217723 (+1548225us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54000d9b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:04.217922 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448642432}
I20260709 12:11:04.185253 13906 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:04.228307 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.230659 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.232348 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.233743 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.241441 10430 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:04.243294 10579 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:04.244882 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.245852 10583 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:04.239331 13793 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:04.248027 13429 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:04.248394 13943 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.248587 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:04.249038 13943 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:04.249626 13793 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:04.250420 13690 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:04.250797 13793 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.248710 13429 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:04.249737 13943 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:04.196350 13944 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:04.244997 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.255541 10582 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:04.169389 13870 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:11:04.211140 13786 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:04.238314 13942 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:04.265758 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.266341 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2145) took 1289 ms. Trace:
I20260709 12:11:04.266809 10505 rpcz_store.cc:276] 0709 12:11:02.977174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.977395 (+   221us) service_pool.cc:224] Handling call
0709 12:11:04.266092 (+1288697us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b5400413a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:04.266265 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888402176}
I20260709 12:11:04.269058 13429 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.266000 13942 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:04.250866 13690 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:04.257090 13763 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:04.324671 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 128849us
I20260709 12:11:04.334659 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.338749 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.343534 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:04.346535 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 121973us
I20260709 12:11:04.347922 13944 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:04.349347 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.350092 11217 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:04.280597 13942 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.353012 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:04.325338 13690 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:04.353813 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:04.356051 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:04.363382 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:04.240736 13819 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.366729 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.367781 13929 consensus_queue.cc:1243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:11:04.375856 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 115832us
I20260709 12:11:04.280864 13640 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:04.379459 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.381678 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:04.391042 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.371232 13943 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:04.342353 13763 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:04.401355 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:04.402050 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.411851 10579 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:04.381731 13870 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:04.418943 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:04.426993 10584 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
W20260709 12:11:04.415318 13905 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.790s	user 0.004s	sys 0.000s
I20260709 12:11:04.399204 13441 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:04.399097 13785 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:04.399500 13946 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:04.428206 13936 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.266s	user 0.000s	sys 0.002s
I20260709 12:11:04.428697 13933 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:04.399050 13947 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
W20260709 12:11:04.428597 13917 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.720s	user 0.001s	sys 0.003s
I20260709 12:11:04.407740 13952 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:11:04.401362 13786 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.431510 13957 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:11:04.449352 13951 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:04.408841 13949 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:11:04.441861 13946 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:04.429891 13870 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.479930 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2146) took 1487 ms. Trace:
I20260709 12:11:04.482681 10512 rpcz_store.cc:276] 0709 12:11:02.993012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:03.008055 (+ 15043us) service_pool.cc:224] Handling call
0709 12:11:03.226275 (+218220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:03.253288 (+ 27013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.479659 (+1226371us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b540036e550. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:04.479850 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500844928}
W20260709 12:11:04.485659 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 117513us
I20260709 12:11:04.447045 13441 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:11:04.453143 13933 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:04.490600 13640 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.464831 13954 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:04.488533 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2116) took 1412 ms. Trace:
I20260709 12:11:04.541764 10585 rpcz_store.cc:276] 0709 12:11:03.076134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:03.602454 (+526320us) service_pool.cc:224] Handling call
0709 12:11:03.627755 (+ 25301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:03.662730 (+ 34975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:03.662766 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:03.663422 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 12:11:03.799064 (+135642us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:03.799112 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.484547 (+685435us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b5400334d54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:04.484605 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:04.484663 (+    58us) raft_consensus.cc:1685] finished
0709 12:11:04.484675 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:04.488452 (+  3777us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":481,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"spinlock_wait_cycles":79316352,"thread_start_us":135195,"threads_started":1,"wal-append.queue_time_us":183661}
I20260709 12:11:04.481052 13946 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:04.499181 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 98398.5us
I20260709 12:11:04.477095 13816 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:04.507102 13951 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:04.507414 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1773) took 1603 ms. Trace:
I20260709 12:11:04.575995 10504 rpcz_store.cc:276] 0709 12:11:02.904175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.911133 (+  6958us) service_pool.cc:224] Handling call
0709 12:11:02.936933 (+ 25800us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:11:02.943547 (+  6614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:02.943599 (+    52us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:11:02.973183 (+ 29584us) log.cc:844] Serialized 78 byte log entry
0709 12:11:04.227127 (+1253944us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b18002c1be0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:04.343577 (+116450us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:04.369745 (+ 26168us) log.cc:844] Serialized 14 byte log entry
0709 12:11:04.370195 (+   450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.370357 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:04.507057 (+136700us) raft_consensus.cc:1685] finished
0709 12:11:04.507174 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:04.507341 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18609,"prepare.queue_time_us":14794,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":3388,"raft.queue_time_us":27365,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":15449,"spinlock_wait_cycles":32081536,"thread_start_us":115834,"threads_started":1,"wal-append.queue_time_us":1449105}
I20260709 12:11:04.509747 13870 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:04.494307 13957 consensus_queue.cc:1243] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:04.513266 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 312) took 1354 ms. Trace:
I20260709 12:11:04.580191 10433 rpcz_store.cc:276] 0709 12:11:03.164585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:03.165539 (+   954us) service_pool.cc:224] Handling call
0709 12:11:03.165941 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:03.215437 (+ 49496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:03.215492 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:03.240224 (+ 24732us) log.cc:844] Serialized 26 byte log entry
0709 12:11:03.649384 (+409160us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:03.781253 (+131869us) log.cc:844] Serialized 14 byte log entry
0709 12:11:03.781507 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:03.781601 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:04.256927 (+475326us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:04.512938 (+256011us) raft_consensus.cc:1685] finished
0709 12:11:04.513050 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:04.513193 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":197416,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":76979,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":30784,"spinlock_wait_cycles":319582464,"thread_start_us":462967,"threads_started":2,"wal-append.queue_time_us":418583}
W20260709 12:11:04.514424 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.515363 13942 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.516387 13787 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.518321 13690 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.488749 13959 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:11:04.529162 13967 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:04.531093 10403 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
W20260709 12:11:04.531598 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 148401us
I20260709 12:11:04.535786 13962 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:04.537596 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:04.538434 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:04.538512 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:04.525292 13798 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.560s	user 0.013s	sys 0.001s
I20260709 12:11:04.545941 13763 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:04.539476 13639 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:04.555433 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:04.539960 13933 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.585357 11220 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:04.587239 10509 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:04.587635 10508 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:04.590408 10507 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:04.594485 10506 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:04.594776 10505 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:04.594156 10512 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10.
I20260709 12:11:04.551256 13794 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.607542 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:04.608445 10504 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:04.610723 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 79882.9us
I20260709 12:11:04.611768 10403 leader_election.cc:400] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:04.612077 10403 leader_election.cc:403] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:11:04.613005 10403 leader_election.cc:400] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6.
I20260709 12:11:04.613292 10403 leader_election.cc:403] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:04.613381 13639 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.615593 13954 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:04.616626 13954 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 5. Current term is 6.
I20260709 12:11:04.599452 13970 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:11:04.616751 13951 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.617940 13952 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:04.618321 13952 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:04.619572 13952 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.608726 13967 consensus_queue.cc:1243] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:11:04.600346 13962 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.621361 13952 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:04.632751 13690 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:04.616638 13787 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:04.621762 13899 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.970s	user 0.002s	sys 0.001s
I20260709 12:11:04.622745 13942 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.623180 13968 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:11:04.635288 13968 consensus_queue.cc:1243] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:11:04.603458 13798 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.637686 13798 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.599886 13816 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:04.639588 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.641752 13816 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.642304 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.642997 10587 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:04.638492 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.654237 10587 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:04.655987 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:04.657105 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:04.657970 10504 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:04.667117 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:04.672940 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:04.674000 10433 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.681730 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:04.682581 10430 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:11:04.683246 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 75154us
I20260709 12:11:04.665117 13948 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:04.687855 13962 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.689307 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:04.690828 10432 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:04.691267 10432 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:04.692055 10432 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.692476 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.693221 10432 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:04.693838 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.693967 13763 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:04.695530 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:04.694907 13763 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:11:04.696343 10504 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:04.697184 13763 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:04.697618 13763 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:04.699558 13976 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:04.700209 13916 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.034s	user 0.002s	sys 0.002s
I20260709 12:11:04.700703 13947 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:04.698696 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:04.701951 13945 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:04.702085 10437 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:04.702507 13945 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:04.703037 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.682623 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:04.704066 10505 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:04.705557 13927 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:04.705966 13927 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:04.703583 13945 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.702805 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:04.707414 13945 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:04.708017 13967 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:04.709982 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:04.710927 13911 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.044s	user 0.005s	sys 0.003s
I20260709 12:11:04.711342 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.712030 11218 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:04.713032 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:04.713376 13915 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.002s	user 0.001s	sys 0.002s
I20260709 12:11:04.715098 10506 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:11:04.715220 13948 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:04.711372 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:04.718966 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:04.719729 10436 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:04.729238 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2183) took 2030 ms. Trace:
I20260709 12:11:04.729697 10580 rpcz_store.cc:276] 0709 12:11:02.699322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:03.515174 (+815852us) service_pool.cc:224] Handling call
0709 12:11:03.515570 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:03.671235 (+155665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.728968 (+1057733us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b5400276ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:04.729173 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344852352}
W20260709 12:11:04.731139 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44202.8us
I20260709 12:11:04.731743 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:04.732400 10435 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.733851 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.735430 10585 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
W20260709 12:11:04.759120 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47837.6us
I20260709 12:11:04.761162 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:04.682641 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:04.783588 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:04.783967 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:04.787092 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 104573us
I20260709 12:11:04.787230 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:04.789498 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:04.789932 10429 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:04.788486 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:04.790773 10436 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:04.787928 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:04.793772 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:04.810360 10583 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:04.811231 10582 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:04.691080 13826 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:04.814792 10505 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:11:04.819854 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:04.829471 10432 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:04.803292 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.716424 13824 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:04.832118 13690 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:04.857097 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:04.761307 13924 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:04.857097 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:04.859315 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 75841.2us
I20260709 12:11:04.873963 11217 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:04.874261 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.706573 13946 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.736105 13944 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:04.836056 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:04.735785 13639 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:04.866899 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.869379 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.795563 10430 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:04.876341 10436 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:04.706845 13927 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.809808 13968 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:04.791582 13640 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:04.894441 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.698887 13763 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.910277 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:04.883304 13955 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:11:04.912099 10586 pending_rounds.cc:85] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 2
I20260709 12:11:04.912416 10586 pending_rounds.cc:107] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:11:04.912671 10586 raft_consensus.cc:2889] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:04.829294 13787 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:04.897184 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.917117 10585 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.897679 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:04.918313 10583 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:04.897696 10506 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:04.898912 10505 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:04.764786 13978 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.119s
I20260709 12:11:04.767477 13785 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 12:11:04.858436 13947 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:04.888200 10429 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
W20260709 12:11:04.906786 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.737090 13856 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 12:11:04.862901 13826 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:04.919217 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.808725 13962 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:11:04.735563 13933 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.895790 13976 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:04.900637 13945 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:04.942054 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.859797 13824 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:04.900902 13639 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:04.950500 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2127) took 2395 ms. Trace:
I20260709 12:11:04.981269 10510 rpcz_store.cc:276] 0709 12:11:02.554714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:02.786286 (+231572us) service_pool.cc:224] Handling call
0709 12:11:02.853264 (+ 66978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:03.562776 (+709512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.950157 (+1387381us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540038f9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:04.950422 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629741952}
W20260709 12:11:04.983924 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.964648 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:04.957165 10438 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:04.969990 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:04.965184 10433 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:04.974165 11218 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 11. Ops: []
I20260709 12:11:04.975241 10583 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:04.968925 10585 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:04.980505 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.926308 13967 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:05.022440 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.939322 13968 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:04.939271 13948 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:04.944360 13927 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:05.031111 10509 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:04.976468 13498 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:04.897117 13944 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:04.945395 13787 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:04.968655 13918 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.174s	user 0.001s	sys 0.002s
I20260709 12:11:04.943149 13763 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:05.028307 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:04.990945 13886 raft_consensus.cc:2764] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:05.006575 13214 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:04.976051 13946 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:04.962036 13933 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:05.026144 13945 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:05.050710 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.023640 13934 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.027522 13947 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:11:05.032698 13982 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.072046 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.033510 13968 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:05.055033 13214 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:05.081780 13968 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:05.082378 13974 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:05.082510 13214 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.084053 13214 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:05.085407 13913 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.375s	user 0.001s	sys 0.004s
I20260709 12:11:05.091682 13968 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.092715 13639 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:05.095458 13982 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.096433 13787 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.097002 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.097472 13755 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:05.098105 13982 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.097816 13933 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:05.095893 13945 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:05.099493 13886 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:05.099939 13945 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.100296 13944 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.100425 13945 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:11:05.096227 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.964663 13826 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.056402 13967 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:05.104802 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.104331 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.080405 13924 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:05.113595 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.097872 13755 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:05.118834 10506 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:05.124508 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.082968 13974 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.131543 10438 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:05.136109 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2150) took 1117 ms. Trace:
I20260709 12:11:05.136577 10438 rpcz_store.cc:276] 0709 12:11:04.018945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:04.040666 (+ 21721us) service_pool.cc:224] Handling call
0709 12:11:04.057853 (+ 17187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:04.956659 (+898806us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400082b20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a54a1d 00007f2c92a3b1a4 00007f2c92abd447 00007f2c92ad5fe7 00007f2c92ad77de 00007f2c92ae1b49 00007f2c92ae2416 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419
0709 12:11:05.135712 (+179053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.136026 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639096064}
I20260709 12:11:05.143584 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.140982 10435 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:05.105065 13856 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.148171 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:05.150161 10507 raft_consensus.cc:606] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:05.148957 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.152032 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:05.152704 10580 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:05.155236 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.164947 10580 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:05.112986 13982 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:05.142910 13952 consensus_meta.cc:287] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 12:11:05.130415 13924 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.133718 13954 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:05.145895 13947 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:05.141988 13974 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:05.213264 13755 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.141845 13784 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 12:11:05.217720 13909 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.572s	user 0.002s	sys 0.002s
I20260709 12:11:05.179085 13862 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.223441 13862 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.224673 13953 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:05.225142 13953 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:05.225826 13862 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.184010 13856 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.229621 13856 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:05.229254 13974 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.147001 13933 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:11:05.144354 13870 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:11:05.231027 13885 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:05.231436 13885 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:05.081722 13934 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.232390 13885 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.233512 13974 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:05.233806 13934 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.234117 13885 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:05.193720 13924 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:05.193271 13982 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:05.199972 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.200924 13976 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:05.180719 13967 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:05.204631 13441 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.242892 13441 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.210361 13945 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.212019 13824 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.250066 13441 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:05.250684 13945 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:05.252374 13982 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:05.254251 13912 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.457s	user 0.004s	sys 0.000s
I20260709 12:11:05.263691 11216 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:05.266222 13968 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:05.275449 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:05.277760 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 407878us
I20260709 12:11:05.293897 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:05.294343 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 267644us
I20260709 12:11:05.301923 10552 leader_election.cc:400] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10.
I20260709 12:11:05.302485 10552 leader_election.cc:403] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:05.305982 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:05.306792 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 470841us
I20260709 12:11:05.312683 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:05.314409 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.317284 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.226388 13953 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.321333 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:05.327625 10435 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:05.298592 10553 leader_election.cc:336] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.581s (SENT)
I20260709 12:11:05.328557 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:05.332253 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.335122 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.336519 10434 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:11:05.337090 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.337006 10434 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:05.337582 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.350554 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:05.351142 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.348313 10434 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.386161 10434 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:05.380580 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.351591 10579 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:05.350248 13968 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.315481 13214 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:11:05.329536 13982 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10.
I20260709 12:11:05.333590 13862 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:11:05.407650 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:05.408850 10580 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:05.409863 10583 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:11:05.348994 13952 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:05.319562 13934 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:11:05.352036 13791 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:05.342393 13773 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:11:05.416960 11221 raft_consensus.cc:2410] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:11:05.420090 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:05.410916 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59854.7us
I20260709 12:11:05.432976 11218 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:05.433389 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.416177 13870 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:05.346709 13996 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:11:05.445502 10580 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:11:05.453195 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 304319us
I20260709 12:11:05.454141 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.456041 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:05.457984 10586 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.017s	sys 0.010s
W20260709 12:11:05.458511 10586 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.017s	sys 0.010s
I20260709 12:11:05.428315 13791 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:05.462551 10434 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:11:05.410394 13968 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.470018 10434 pending_rounds.cc:85] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:05.470400 10434 pending_rounds.cc:107] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:11:05.470781 10434 raft_consensus.cc:2889] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:05.440937 13952 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:05.440783 13978 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:05.498488 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.442493 13868 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 12:11:05.535730 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62794.7us
W20260709 12:11:05.536839 10553 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.503s (SENT)
I20260709 12:11:05.468850 13948 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:05.460997 13870 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.473731 13791 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:05.544972 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:05.499197 13795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.824s	user 0.022s	sys 0.026s
I20260709 12:11:05.479312 13952 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:05.545635 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.554322 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.555979 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:05.551122 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.555573 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:05.564090 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.565601 10508 raft_consensus.cc:2368] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:05.565519 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.566346 10434 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:05.534746 13967 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:11:05.569983 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.476320 13704 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:11:05.580008 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:05.581933 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.583736 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.542121 13996 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.591472 11212 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:05.551384 13795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.875s	user 0.023s	sys 0.026s
I20260709 12:11:05.585994 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:05.593461 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:05.594049 10579 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:05.593919 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:05.586736 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.542590 13868 consensus_queue.cc:1243] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:11:05.595320 10435 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10.
I20260709 12:11:05.546762 13978 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.598673 13978 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.600366 13794 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:05.542001 13773 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:11:05.574455 13948 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:05.603361 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.603549 13948 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.605957 13948 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:05.606859 13991 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:11:05.611320 13998 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:05.611758 14003 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:11:05.611850 13998 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:05.613018 13998 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.614106 13996 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.616602 13763 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:05.617807 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:05.618407 13755 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:05.619350 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:05.620576 10505 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:05.621003 10509 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:05.627635 11216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 431) took 1309 ms. Trace:
I20260709 12:11:05.617039 13763 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:05.629189 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36746 (request call id 487) took 1238 ms. Trace:
I20260709 12:11:05.629582 13763 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.629622 10584 rpcz_store.cc:276] 0709 12:11:04.390370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:04.395467 (+  5097us) service_pool.cc:224] Handling call
0709 12:11:05.629002 (+1233535us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b5400335c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:05.629116 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16509824}
I20260709 12:11:05.632855 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.634143 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.637437 13968 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:05.628130 11216 rpcz_store.cc:276] 0709 12:11:04.318334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:04.319110 (+   776us) service_pool.cc:224] Handling call
0709 12:11:05.627404 (+1308294us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400591854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:05.627542 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114041728}
I20260709 12:11:05.630896 13763 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:05.641103 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47288.1us
W20260709 12:11:05.578699 13927 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.540s	user 0.013s	sys 0.000s
W20260709 12:11:05.655186 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.655268 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.656059 10512 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:05.657447 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.658390 10429 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:05.662111 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.662971 10435 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10.
I20260709 12:11:05.667412 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.668550 10436 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:05.648046 11218 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:05.651854 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:05.617107 13996 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.668262 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.673265 10435 raft_consensus.cc:2368] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 5. Current term is 6.
I20260709 12:11:05.648823 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.670086 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.675668 10432 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:05.670507 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.671646 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.683743 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.684851 10429 raft_consensus.cc:2368] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8.
I20260709 12:11:05.685086 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.654274 13968 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:05.686874 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:05.689149 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.691524 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.693302 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.696916 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.695182 13639 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:05.706506 13639 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:05.698539 13903 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:05.727259 13639 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:05.714653 10552 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.728837 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:05.721081 13795 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:05.730535 13795 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:05.731545 13795 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.732899 13795 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:05.734094 14006 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:05.734973 14006 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:05.735407 14006 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:05.735771 14006 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:05.736277 13885 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.006s
I20260709 12:11:05.723712 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:05.739344 14012 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:11:05.739648 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.739868 10584 raft_consensus.cc:2368] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:05.658203 13787 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:11:05.698931 13819 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:05.741295 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.734651 13927 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:05.749106 13927 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:05.652765 13870 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.751595 13927 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:05.742339 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.754504 13819 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.694949 13660 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:11:05.756708 13660 consensus_queue.cc:1243] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:11:05.710741 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:05.765214 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.749147 13958 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:05.767925 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.772150 10584 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:05.775223 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:05.777532 10584 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:05.776539 10579 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:11:05.778466 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:05.780014 11218 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:05.784868 10436 pending_rounds.cc:85] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 5
I20260709 12:11:05.785226 10436 pending_rounds.cc:107] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:11:05.785509 10436 raft_consensus.cc:2889] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:05.786375 10585 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:05.793332 11218 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:05.797013 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:05.798570 10579 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
W20260709 12:11:05.817852 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.789s (SENT)
I20260709 12:11:05.818523 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:05.791646 13903 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.834004 10437 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:11:05.830891 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.805673 13785 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:05.838939 10429 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:05.833156 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.844400 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.841809 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 148633us
I20260709 12:11:05.804934 13958 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.805507 13824 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:05.850935 13824 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.852808 13697 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:11:05.855901 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:05.855921 13787 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:05.839963 13996 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:05.863086 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.861647 10429 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 12:11:05.863884 10437 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10.
I20260709 12:11:05.845876 13785 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:05.880044 13704 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:05.829005 13819 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:05.887169 13968 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:05.887617 13968 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.888263 13640 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:05.888460 13968 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.889822 13704 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.890045 13958 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.890868 13704 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.891361 13793 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:05.891484 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:05.892371 10509 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:05.892765 13710 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:05.893187 13710 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.883883 10506 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.880604 13927 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.880532 13785 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:05.908046 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.883145 13787 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.913714 10509 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
W20260709 12:11:05.925662 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.929697 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.864960 13429 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 12:11:05.934007 11178 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.793s (SENT) [suppressed 30 similar messages]
W20260709 12:11:05.906114 13924 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.660s	user 0.014s	sys 0.002s
I20260709 12:11:05.938086 10434 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:05.941947 13429 consensus_queue.cc:1243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:11:05.943506 14017 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
W20260709 12:11:05.945557 14024 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:11:05.939425 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:05.947433 13793 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:05.947685 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2144) took 1078 ms. Trace:
I20260709 12:11:05.948031 13793 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:05.948133 10586 rpcz_store.cc:276] 0709 12:11:04.868780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:04.911433 (+ 42653us) service_pool.cc:224] Handling call
0709 12:11:04.911825 (+   392us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:11:05.458997 (+547172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.459048 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:11:05.460806 (+  1758us) log.cc:844] Serialized 78 byte log entry
0709 12:11:05.826346 (+365540us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:11:05.826409 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.826497 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:05.947388 (+120891us) raft_consensus.cc:1685] finished
0709 12:11:05.947413 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.947596 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3300,"raft.queue_time_us":70448,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":750473,"threads_started":2,"wal-append.queue_time_us":358144}
I20260709 12:11:05.949219 13793 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:05.961627 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:05.947157 10513 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:11:05.977397 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:05.978142 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.987169 13982 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.987797 13982 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.990162 13982 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:05.991822 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.992549 10432 raft_consensus.cc:2376] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6. Re-sending same reply.
I20260709 12:11:05.993606 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:05.994320 10432 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:05.994239 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:05.901898 13903 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.011734 13787 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.011183 14006 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.026824 13826 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:06.027591 13826 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:06.015199 13824 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:05.916256 13791 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:06.019026 13949 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:11:05.960587 13924 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:05.918944 13927 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.081449 13924 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:06.021047 13498 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:06.082125 13927 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:06.082412 13498 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:05.937469 13710 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.084060 13498 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:06.023670 13952 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:06.051766 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:06.087852 10507 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.053092 13791 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:06.087802 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.096055 10432 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:11:06.073228 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70348.7us
I20260709 12:11:06.076601 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:06.078974 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:06.098157 10509 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:06.086664 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.099145 10403 leader_election.cc:400] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:06.099501 10403 leader_election.cc:403] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:06.101228 13640 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:06.104247 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:06.043985 14006 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:06.045527 13826 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:06.136233 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:06.107993 13927 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:06.144236 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:06.123718 13924 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:06.154968 11178 leader_election.cc:400] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10.
I20260709 12:11:06.155501 11178 leader_election.cc:403] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:11:06.151738 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.157773 10553 leader_election.cc:400] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8.
I20260709 12:11:06.158128 10553 leader_election.cc:403] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:06.160593 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:06.169328 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:06.162237 13840 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:06.142585 13949 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:11:06.135707 13785 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:06.189872 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 383626us
I20260709 12:11:06.172958 13791 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8.
I20260709 12:11:06.196846 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:06.200441 10553 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.175s (SENT)
I20260709 12:11:06.202001 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.207237 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:06.194051 13786 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:11:06.216833 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.218339 10432 raft_consensus.cc:2410] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:06.155014 10507 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:06.223948 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.230279 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:06.231412 14006 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:06.222059 13639 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.233793 10508 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:06.194229 13972 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:11:06.225752 13786 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:06.235520 13786 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.236040 13786 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.208122 13784 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:06.239889 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:06.240733 10509 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:06.197661 13640 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 12:11:06.241467 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.241765 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:06.242823 14026 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:11:06.245043 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:06.239883 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.234584 13639 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:06.276612 10432 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 6: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:06.282181 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:06.285828 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:06.291795 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:06.295230 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65052.5us
I20260709 12:11:06.296324 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:06.298821 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:06.300578 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:06.302865 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.306774 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.316425 10432 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10.
I20260709 12:11:06.322219 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:06.330684 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.332626 10432 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:06.320569 13927 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:06.335160 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.336019 10403 leader_election.cc:400] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:06.337057 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:06.336045 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 358756us
I20260709 12:11:06.324790 13924 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:06.341951 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:06.351938 14028 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:11:06.352842 10583 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:11:06.353369 14030 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:11:06.328322 13791 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:06.355073 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.326531 13886 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:06.355741 10436 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:06.356035 13886 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:06.348927 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:06.356691 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.356976 10580 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:06.357784 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.347378 13948 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:06.359059 13948 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:06.359663 13948 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.362207 13948 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:06.363869 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:06.365069 10436 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:06.365137 13998 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:06.366272 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.367444 10436 raft_consensus.cc:2410] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:06.368438 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:06.369971 13968 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:06.350216 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:06.371500 10432 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:06.380789 10436 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:11:06.415943 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 176334us
I20260709 12:11:06.407612 13966 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:06.381278 13927 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:06.412828 13763 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:06.460053 10582 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:06.461503 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:06.469686 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 127834us
I20260709 12:11:06.471490 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:06.480479 10437 raft_consensus.cc:1217] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[6.3-10.5]   Dedup: 6.3->[8.4-10.5]
I20260709 12:11:06.481101 10437 pending_rounds.cc:85] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:06.481356 10437 pending_rounds.cc:107] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 12:11:06.481602 10437 raft_consensus.cc:2889] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:06.482486 10430 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:11:06.450920 13755 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:06.446039 13967 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:06.433200 13964 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:11:06.431007 13793 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 12:11:06.501286 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:06.501848 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.453102 13763 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:06.509419 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93870.3us
I20260709 12:11:06.494040 13967 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:06.447991 13966 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:06.518828 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:06.507221 13840 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10.
W20260709 12:11:06.581039 10553 leader_election.cc:336] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.551s (SENT)
I20260709 12:11:06.586484 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:06.591063 13441 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:06.601691 10512 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:11:06.602134 10512 pending_rounds.cc:85] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 8
I20260709 12:11:06.602365 10512 pending_rounds.cc:107] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:11:06.602581 10512 raft_consensus.cc:2889] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:06.543452 14042 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:06.539757 13826 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:11:06.599072 13441 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:06.617463 14042 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.643009 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:06.643946 10582 raft_consensus.cc:2368] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9.
I20260709 12:11:06.646135 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:06.647089 11177 leader_election.cc:400] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9.
I20260709 12:11:06.647529 11177 leader_election.cc:403] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:06.648893 11217 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:06.653182 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:06.648522 14041 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:11:06.638231 14042 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.650920 13943 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.640872 13966 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:06.681136 11217 raft_consensus.cc:2368] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
W20260709 12:11:06.674468 14036 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:11:06.655517 14062 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:06.714535 14046 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:11:06.717095 14051 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:11:06.719548 14053 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:11:06.763815 14042 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:06.767171 14042 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:06.862746 13870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.007s	sys 0.000s
W20260709 12:11:06.863411 13870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.008s	sys 0.000s
W20260709 12:11:06.866093 14055 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.210s	user 0.003s	sys 0.000s
W20260709 12:11:07.018358 13998 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.653s	user 0.016s	sys 0.002s
I20260709 12:11:07.028403 13763 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:06.676149 13441 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.033980 13441 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:07.045423 10436 pending_rounds.cc:85] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:07.045900 10436 pending_rounds.cc:107] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:11:07.046181 10436 raft_consensus.cc:2889] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:07.050822 10434 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 321) took 1483 ms (client timeout 1793 ms). Trace:
W20260709 12:11:07.051395 10434 rpcz_store.cc:269] 0709 12:11:05.567918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.568060 (+   142us) service_pool.cc:224] Handling call
0709 12:11:07.050623 (+1482563us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54001676d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:07.050743 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839928192}
I20260709 12:11:07.053128 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2188) took 1136 ms. Trace:
I20260709 12:11:07.053537 10434 rpcz_store.cc:276] 0709 12:11:05.917202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.052483 (+1135281us) service_pool.cc:224] Handling call
0709 12:11:07.052749 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.052904 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.052920 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.052949 (+    29us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:07.052969 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.053002 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.053082 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:07.055573 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2189) took 1136 ms. Trace:
I20260709 12:11:07.055826 10434 rpcz_store.cc:276] 0709 12:11:05.919374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.054900 (+1135526us) service_pool.cc:224] Handling call
0709 12:11:07.055228 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.055363 (+   135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.055378 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.055410 (+    32us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:07.055428 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.055460 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.055532 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:07.057343 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2190) took 1132 ms. Trace:
I20260709 12:11:07.057554 10434 rpcz_store.cc:276] 0709 12:11:05.925129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.056690 (+1131561us) service_pool.cc:224] Handling call
0709 12:11:07.056983 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.057141 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.057158 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.057185 (+    27us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:07.057206 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.057236 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.057307 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:07.079805 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1808) took 1349 ms. Trace:
I20260709 12:11:07.080350 10508 rpcz_store.cc:276] 0709 12:11:05.730649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.738788 (+  8139us) service_pool.cc:224] Handling call
0709 12:11:05.739184 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:06.234178 (+494994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.079462 (+845284us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400412654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:07.079716 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137020288}
I20260709 12:11:07.098182 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2184) took 1189 ms. Trace:
I20260709 12:11:07.098738 10430 rpcz_store.cc:276] 0709 12:11:05.908524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.417147 (+508623us) service_pool.cc:224] Handling call
0709 12:11:06.417670 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:06.482904 (+ 65234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.097833 (+614929us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b5400320354. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:07.098093 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122541824}
I20260709 12:11:07.100709 10430 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 12:11:07.101079 10430 pending_rounds.cc:85] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:07.101323 10430 pending_rounds.cc:107] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:11:07.101569 10430 raft_consensus.cc:2889] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:07.101889 10430 pending_rounds.cc:107] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:11:07.102928 10430 raft_consensus.cc:2889] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:07.103850 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1710) took 1159 ms. Trace:
I20260709 12:11:07.104332 10430 rpcz_store.cc:276] 0709 12:11:05.944925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.099802 (+1154877us) service_pool.cc:224] Handling call
0709 12:11:07.100180 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.103607 (+  3427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.103796 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:07.106007 10430 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:11:07.106323 10430 pending_rounds.cc:85] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 2
I20260709 12:11:07.106550 10430 pending_rounds.cc:107] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:11:07.106786 10430 raft_consensus.cc:2889] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:07.107110 10430 pending_rounds.cc:107] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:11:07.107409 10430 raft_consensus.cc:2889] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:07.108105 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1713) took 1162 ms. Trace:
I20260709 12:11:07.108430 10430 rpcz_store.cc:276] 0709 12:11:05.945934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.105364 (+1159430us) service_pool.cc:224] Handling call
0709 12:11:07.105667 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.107876 (+  2209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.108058 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:07.109604 10430 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:07.110154 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1714) took 1164 ms. Trace:
I20260709 12:11:07.110471 10430 rpcz_store.cc:276] 0709 12:11:05.946246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.109139 (+1162893us) service_pool.cc:224] Handling call
0709 12:11:07.109444 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.109941 (+   497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.110109 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:07.128306 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.011s	sys 0.014s
W20260709 12:11:07.128808 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.011s	sys 0.014s
W20260709 12:11:07.129112 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 536419us
I20260709 12:11:07.133908 13868 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:11:07.142563 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:07.143860 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.145151 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:07.145826 10508 raft_consensus.cc:606] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:07.147820 11217 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:11:06.676862 13927 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.151831 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.153903 10506 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:07.155229 13785 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.161652 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:07.161666 13972 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:07.162467 10506 raft_consensus.cc:606] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:07.164460  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:07.178781 13868 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Committing config change with OpId 7.6: config changed from index 3 to 6, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:11:07.180493 14040 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.588s	user 0.002s	sys 0.000s
I20260709 12:11:07.182866 13704 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:07.183373 11217 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 3 to 6, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:07.195391 13704 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:07.198781 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2140) took 2331 ms. Trace:
I20260709 12:11:07.199265 10587 rpcz_store.cc:276] 0709 12:11:04.867640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:04.867766 (+   126us) service_pool.cc:224] Handling call
0709 12:11:04.868097 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:05.048501 (+180404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.048529 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:05.049166 (+   637us) log.cc:844] Serialized 52 byte log entry
0709 12:11:05.448921 (+399755us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:05.448976 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.198319 (+1749343us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400334854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:07.198372 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.198423 (+    51us) raft_consensus.cc:1685] finished
0709 12:11:07.198438 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.198703 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6740,"raft.queue_time_us":1422,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":16777,"spinlock_wait_cycles":413484288,"thread_start_us":341599,"threads_started":1,"wal-append.queue_time_us":559485,"wal-append.run_cpu_time_us":2238,"wal-append.run_wall_time_us":1004404}
I20260709 12:11:07.202457 13710 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:07.203619 13868 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41062 (request call id 85) took 1349 ms. Trace:
I20260709 12:11:07.204097 13868 rpcz_store.cc:276] 0709 12:11:05.853804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.854717 (+   913us) service_pool.cc:224] Handling call
0709 12:11:05.903520 (+ 48803us) log.cc:844] Serialized 487 byte log entry
0709 12:11:07.203463 (+1299943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69393,"raft.queue_time_us":58512,"raft.run_cpu_time_us":1836,"raft.run_wall_time_us":56465,"thread_start_us":429908,"threads_started":1,"wal-append.queue_time_us":584013}
W20260709 12:11:07.219381 13964 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.003s	sys 0.001s
W20260709 12:11:07.219995 13964 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.003s	sys 0.001s
I20260709 12:11:07.229115 14006 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.229867 14006 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.231361 13785 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.232122 13785 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.233067 14006 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.233366 13763 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.234133 13763 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.236171 11221 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:07.237251 13763 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.238224 14063 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:07.243551 11221 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:07.245111 13785 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:07.249053 14033 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.023s	sys 0.031s
I20260709 12:11:07.249078 13968 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:11:07.249809 14033 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.023s	sys 0.031s
I20260709 12:11:07.277577 13953 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:07.284590 13824 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:07.285475 13824 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.287367 13984 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.287972 13984 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.288267 13824 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.291666 13984 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.293936 13967 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.297348 10431 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:07.305713 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1719) took 1352 ms. Trace:
I20260709 12:11:07.302213 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 319) took 1738 ms. Trace:
I20260709 12:11:07.339421 10438 rpcz_store.cc:276] 0709 12:11:05.564279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.564392 (+   113us) service_pool.cc:224] Handling call
0709 12:11:05.564686 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.595533 (+ 30847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.595568 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.595613 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:05.595643 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.301911 (+1706268us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b540033ccd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:07.301965 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.302130 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70388992}
W20260709 12:11:07.306322 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.326139 13952 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:07.348954 10551 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.003s	sys 0.003s
W20260709 12:11:07.336325 11212 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.000s	sys 0.016s
I20260709 12:11:07.348603 10431 rpcz_store.cc:276] 0709 12:11:05.953099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.296663 (+1343564us) service_pool.cc:224] Handling call
0709 12:11:07.297110 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:07.305404 (+  8294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.305649 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:07.349819 11212 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.000s	sys 0.017s
W20260709 12:11:07.351189 10551 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.003s	sys 0.004s
I20260709 12:11:07.354130 10431 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:07.354859 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2192) took 1396 ms. Trace:
I20260709 12:11:07.355233 10431 rpcz_store.cc:276] 0709 12:11:05.958715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.350286 (+1391571us) service_pool.cc:224] Handling call
0709 12:11:07.353946 (+  3660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.354591 (+   645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.354799 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:07.353034 13824 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.356513 13984 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.358399 13984 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.353653 13785 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.361105 13785 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.363088 13824 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.365435 13824 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.367101 13785 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.371228 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1809) took 1412 ms. Trace:
I20260709 12:11:07.371896 10431 pending_rounds.cc:85] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:07.372277 10431 pending_rounds.cc:107] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:11:07.372538 10431 raft_consensus.cc:2889] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:07.371661 10504 rpcz_store.cc:276] 0709 12:11:05.959363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.959561 (+   198us) service_pool.cc:224] Handling call
0709 12:11:05.964287 (+  4726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.964571 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:11:06.032436 (+ 67865us) log.cc:844] Serialized 14 byte log entry
0709 12:11:06.032704 (+   268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:06.032776 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:06.032841 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:06.032924 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.370965 (+1338041us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b5400152050. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:07.371152 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67645,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":10255,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4554,"spinlock_wait_cycles":9877376}
W20260709 12:11:07.370832 13943 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.000s
I20260709 12:11:07.377279 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2167) took 1815 ms. Trace:
I20260709 12:11:07.379510 10513 rpcz_store.cc:276] 0709 12:11:05.561594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.605768 (+ 44174us) service_pool.cc:224] Handling call
0709 12:11:05.946779 (+341011us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:05.963820 (+ 17041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.377024 (+1413204us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400152050. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:07.377201 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783317376}
I20260709 12:11:07.381138 13785 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.382529 13785 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.385644 13785 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.381990 13984 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:07.383448 13786 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.147s	user 0.016s	sys 0.003s
I20260709 12:11:07.402652 13784 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:07.379513 13943 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9.
I20260709 12:11:07.415647 11218 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:11:07.416045 11218 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 5
I20260709 12:11:07.416253 11218 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.6
I20260709 12:11:07.416499 11218 raft_consensus.cc:2917] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Aborting config change with OpId 7.6: Aborted: Op aborted by new leader
I20260709 12:11:07.416851 11218 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:07.420192 10585 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.029s	sys 0.031s
W20260709 12:11:07.423895 10585 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.029s	sys 0.031s
I20260709 12:11:07.403275 13786 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.412101 13784 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.432680 13784 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.432917 13786 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.434824 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 329) took 1541 ms. Trace:
I20260709 12:11:07.435269 10512 rpcz_store.cc:276] 0709 12:11:05.893381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.910242 (+ 16861us) service_pool.cc:224] Handling call
0709 12:11:05.910504 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:06.603051 (+692547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.434546 (+831495us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54002f2bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:07.434753 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588720384}
I20260709 12:11:07.439095 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:07.448042 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:07.450121 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:07.450742 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.454134 10433 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:07.454958 11221 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:07.455088 10553 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.866s (SENT)
I20260709 12:11:07.456254 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:07.457221 13984 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.458464 13984 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:07.459928 14039 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.969s	user 0.005s	sys 0.000s
I20260709 12:11:07.461671 13984 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.462257 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2176) took 1567 ms. Trace:
I20260709 12:11:07.465058 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:07.465814 10583 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.475309 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:07.475570 13959 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:07.476265 13959 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.480623 13946 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:07.481323 13946 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.464373 10429 rpcz_store.cc:276] 0709 12:11:05.894664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.894825 (+   161us) service_pool.cc:224] Handling call
0709 12:11:05.895226 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:05.895682 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.895716 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:05.900928 (+  5212us) log.cc:844] Serialized 52 byte log entry
0709 12:11:06.304260 (+403332us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:06.304317 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:06.306310 (+  1993us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.461993 (+1155683us) raft_consensus.cc:1685] finished
0709 12:11:07.462017 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.462187 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":608,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5739,"thread_start_us":352855,"threads_started":1,"wal-append.queue_time_us":512675}
I20260709 12:11:07.487452 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2176) took 1031 ms. Trace:
I20260709 12:11:07.488154 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2180) took 1030 ms. Trace:
I20260709 12:11:07.487879 10578 rpcz_store.cc:276] 0709 12:11:06.456170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.461018 (+  4848us) service_pool.cc:224] Handling call
0709 12:11:06.482923 (+ 21905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:06.485163 (+  2240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:06.485202 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:06.485717 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:11:06.589571 (+103854us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:06.589848 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:06.590087 (+   239us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.487191 (+897104us) raft_consensus.cc:1685] finished
0709 12:11:07.487212 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.487383 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11408,"raft.queue_time_us":3919,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":3560,"thread_start_us":83786,"threads_started":1,"wal-append.queue_time_us":132176}
I20260709 12:11:07.488653 10581 rpcz_store.cc:276] 0709 12:11:06.457424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.458108 (+   684us) service_pool.cc:224] Handling call
0709 12:11:06.477964 (+ 19856us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:06.478388 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:06.478418 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:06.487236 (+  8818us) log.cc:844] Serialized 52 byte log entry
0709 12:11:06.619241 (+132005us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:06.619302 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:06.619397 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.487923 (+868526us) raft_consensus.cc:1685] finished
0709 12:11:07.487946 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.488096 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1396,"raft.queue_time_us":5837,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":1176,"thread_start_us":123914,"threads_started":1,"wal-append.queue_time_us":248631}
I20260709 12:11:07.490715 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:07.491712 10586 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.492733 13946 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.496057 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:07.496693 10504 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.501269 10586 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:07.478809 13959 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.505168 10504 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:07.513983  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:07.515020  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:07.520360 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0b474d9b241d4a698702287ee1bbe3b2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 12:11:07.522884 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2179) took 1623 ms. Trace:
W20260709 12:11:07.525183 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 1.05381e+06us
I20260709 12:11:07.526562 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:07.538107 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2182) took 1072 ms. Trace:
I20260709 12:11:07.538779 10585 rpcz_store.cc:276] 0709 12:11:06.465359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.466641 (+  1282us) service_pool.cc:224] Handling call
0709 12:11:06.467018 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.424390 (+957372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.424424 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.424472 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:07.424507 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.537746 (+113239us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b5400178c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:07.537819 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.538020 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":644304,"raft.queue_time_us":57845,"raft.run_cpu_time_us":767,"raft.run_wall_time_us":644462,"spinlock_wait_cycles":182307584,"thread_start_us":841473,"threads_started":1}
W20260709 12:11:07.540423 11178 leader_election.cc:336] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.785s (SENT)
I20260709 12:11:07.609247 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:07.549872 10334 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.550840 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:07.611037 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60284.1us
I20260709 12:11:07.611578 10334 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.613379 14094 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.553660 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:07.554890 10433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 322) took 1973 ms (client timeout 1838 ms). Trace:
W20260709 12:11:07.615965 10433 rpcz_store.cc:269] 0709 12:11:05.581439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.581596 (+   157us) service_pool.cc:224] Handling call
0709 12:11:07.554659 (+1973063us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400264dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:07.554790 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4239971712}
I20260709 12:11:07.523386 10435 rpcz_store.cc:276] 0709 12:11:05.899420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.899655 (+   235us) service_pool.cc:224] Handling call
0709 12:11:05.901516 (+  1861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.901801 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:06.297850 (+396049us) log.cc:844] Serialized 14 byte log entry
0709 12:11:06.458732 (+160882us) log.cc:844] Serialized 14 byte log entry
0709 12:11:07.379277 (+920545us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18001fa680. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:07.522137 (+142860us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b18001fa680. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c92ad7060 00007f2c92ae9716 00007f2c92afa287 00007f2c92afa208
0709 12:11:07.522383 (+   246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.522457 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:07.522518 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.522620 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.522800 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23216,"prepare.queue_time_us":336702,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":81,"raft.queue_time_us":1072657,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":7752,"spinlock_wait_cycles":11042432,"thread_start_us":541410,"threads_started":2,"wal-append.queue_time_us":209772}
I20260709 12:11:07.565101 13949 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:07.555449 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2186) took 1645 ms. Trace:
I20260709 12:11:07.617312 13949 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:07.566576 13948 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:07.575533 13953 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:07.617496 10437 rpcz_store.cc:276] 0709 12:11:05.909995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.454754 (+544759us) service_pool.cc:224] Handling call
0709 12:11:06.480246 (+ 25492us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:11:07.544985 (+1064739us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:11:07.545021 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:07.545742 (+   721us) log.cc:844] Serialized 52 byte log entry
0709 12:11:07.554970 (+  9228us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:11:07.555038 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.555137 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.555201 (+    64us) raft_consensus.cc:1685] finished
0709 12:11:07.555214 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.555379 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1040721,"raft.queue_time_us":1082797,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"thread_start_us":24517,"threads_started":2,"wal-append.queue_time_us":6863}
I20260709 12:11:07.618475 13953 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.619653 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.619967 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:07.620697 10435 raft_consensus.cc:2376] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7. Re-sending same reply.
I20260709 12:11:07.569839 13924 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:07.593765 14084 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.621486 13924 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:07.621544 14084 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.622258 10435 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:11:07.578047 14052 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:07.623492 13949 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.598932 13967 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.617318 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.599277 13885 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.599064 13886 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.633472 13886 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:07.634863 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 129830us
I20260709 12:11:07.635012 13939 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:07.635774 13939 consensus_queue.cc:1243] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:07.635864 13886 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:07.592450 10332 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 232) took 18016 ms (client timeout 15000 ms). Trace:
W20260709 12:11:07.638554 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98182.8us
I20260709 12:11:07.640529 10583 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:07.614261 10334 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.622795 13924 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.623313 14052 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.594805 14083 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.595556 14086 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.620906 13953 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.625945 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.629314 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:07.624341 13967 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.656376 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:07.631793 10504 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:11:07.624032 13949 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.636138 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:07.642354 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:07.642839 13942 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:07.644116 10553 leader_election.cc:400] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
I20260709 12:11:07.650086 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.669346 10553 leader_election.cc:403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:11:07.670485 10433 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.649825 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:07.651465 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:07.636623 10332 rpcz_store.cc:269] 0709 12:10:49.575672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:49.575926 (+   254us) service_pool.cc:224] Handling call
0709 12:11:07.592342 (+18016416us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:03.708416 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:03.709489 (+  1073us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:03.709559 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:11:03.709585 (+    26us) tablet.cc:662] Decoding operations
0709 12:11:03.713057 (+  3472us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:03.713167 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:03.713222 (+    55us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:11:03.763703 (+ 50481us) tablet.cc:701] Row locks acquired
0709 12:11:03.763767 (+    64us) write_op.cc:260] PREPARE: finished
0709 12:11:03.763977 (+   210us) write_op.cc:270] Start()
0709 12:11:03.768718 (+  4741us) write_op.cc:276] Timestamp: P: 1783599063763903 usec, L: 0
0709 12:11:03.768752 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:11:03.865206 (+ 96454us) log.cc:844] Serialized 8598 byte log entry
0709 12:11:04.162952 (+297746us) op_driver.cc:464] REPLICATION: finished
0709 12:11:04.393957 (+231005us) write_op.cc:301] APPLY: starting
0709 12:11:04.411268 (+ 17311us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:04.491318 (+ 80050us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:04.491490 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:04.635108 (+143618us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:04.635540 (+   432us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 12:11:04.635591 (+    51us) write_op.cc:312] APPLY: finished
0709 12:11:04.636108 (+   517us) log.cc:844] Serialized 152 byte log entry
0709 12:11:04.637158 (+  1050us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:04.637406 (+   248us) write_op.cc:454] Released schema lock
0709 12:11:04.637491 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:04.691123 (+ 53632us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:04.691160 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3094767,"reported_tablets":52,"spinlock_wait_cycles":10368,"child_traces":[["op",{"apply.queue_time_us":196054,"apply.run_cpu_time_us":8431,"apply.run_wall_time_us":284291,"num_ops":22,"prepare.queue_time_us":471903,"prepare.run_cpu_time_us":17819,"prepare.run_wall_time_us":224231,"raft.queue_time_us":142868,"raft.run_cpu_time_us":12602,"raft.run_wall_time_us":224971,"replication_time_us":384943,"thread_start_us":682192,"threads_started":4,"wal-append.queue_time_us":34998,"wal-append.run_cpu_time_us":31242,"wal-append.run_wall_time_us":1737035}]]}
I20260709 12:11:07.614251 14094 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:07.656844 10429 raft_consensus.cc:2410] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:11:07.658792 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.660054 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:07.665876 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.661547 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:07.679955 10435 raft_consensus.cc:2368] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9.
I20260709 12:11:07.682224 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.661636 10334 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.687762 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.691555 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:07.692399 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.693128 10583 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:07.703905 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.704661 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.644157 13984 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.718015 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.656463 13763 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.728822 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.721894 10433 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:07.668068 13948 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:07.666659 13824 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.733318 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.668744 13953 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.643286 13885 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.664412 13943 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
W20260709 12:11:07.708009 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08003525e0 after lost signal to thread 9264
I20260709 12:11:07.715204 14006 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:07.752480 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.754343 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:07.757923 10367 debug-util.cc:398] Leaking SignalData structure 0x7b080003b7a0 after lost signal to thread 10370
W20260709 12:11:07.760495 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08003b95e0 after lost signal to thread 10462
W20260709 12:11:07.761955 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08003f76e0 after lost signal to thread 10537
W20260709 12:11:07.763706 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08000cd5c0 after lost signal to thread 10611
W20260709 12:11:07.763635 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 127575us
W20260709 12:11:07.764248 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc100 after lost signal to thread 10612
I20260709 12:11:07.766134 10583 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:07.724470 10334 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:11:07.771003 10367 debug-util.cc:398] Leaking SignalData structure 0x7b0800145820 after lost signal to thread 11248
W20260709 12:11:07.771536 10367 debug-util.cc:398] Leaking SignalData structure 0x7b080038e4c0 after lost signal to thread 11249
I20260709 12:11:07.771883 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:07.772181 10334 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:11:07.773046 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08001cae80 after lost signal to thread 13999
W20260709 12:11:07.773487 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08003660a0 after lost signal to thread 14004
W20260709 12:11:07.773878 10367 debug-util.cc:398] Leaking SignalData structure 0x7b0800340d80 after lost signal to thread 14020
I20260709 12:11:07.656890 14083 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.776249 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1715) took 1829 ms. Trace:
I20260709 12:11:07.714272 13801 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:11:07.776409 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:07.777670 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 133730us
I20260709 12:11:07.773845 10334 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.778198 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.780149 10430 rpcz_store.cc:276] 0709 12:11:05.946523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.111168 (+1164645us) service_pool.cc:224] Handling call
0709 12:11:07.111494 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:07.775920 (+664426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.775952 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.775992 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:07.776021 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.776084 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.776190 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":482556,"raft.queue_time_us":461017,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":196322,"threads_started":1}
I20260709 12:11:07.788609 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:07.733346 13785 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
I20260709 12:11:07.742345 13886 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.793009 10334 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 7, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:07.733460 13984 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.801560 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:07.776386 13939 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:07.808526 10433 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:07.809767 10433 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:07.775287 13755 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.741701 13763 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.839500 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:07.840677 11221 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:07.664615 14086 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.821426 10334 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 9, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 9 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:07.786454 13942 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:07.785913 14006 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:07.823940 13885 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.779417 14084 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.753479 13824 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.860992 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:07.811873 13784 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:11:07.826783 13984 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.802917 13886 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.775755 13948 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:07.832867 14083 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:07.896461 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 236496us
I20260709 12:11:07.836745 13949 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.838054 14052 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.840570 13763 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:07.811326 13755 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.853019 13942 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:07.856740 10334 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:07.824779 13801 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:07.909797 14087 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.865959 10404 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:07.946089 10404 leader_election.cc:403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:07.823935 13939 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:07.866670 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.854409 14000 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.758152 13953 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.862651 13885 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.862308 14090 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.877154 13946 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:07.864131 10437 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:07.949419 10430 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:07.950809 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:07.965296 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 275232us
I20260709 12:11:07.921038 14091 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.870240 14095 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:11:07.925319 13967 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:07.922282 13949 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.951575 14084 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:07.973919 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.975093 10433 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:11:07.964076 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 177324us
I20260709 12:11:07.967613 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1724) took 2013 ms. Trace:
I20260709 12:11:07.958572 14083 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:07.970260 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:07.957142 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 95532.5us
I20260709 12:11:07.964725 14052 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:07.942485 13801 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:07.947736 10334 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:11:07.976320 10438 rpcz_store.cc:276] 0709 12:11:05.954371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.340689 (+1386318us) service_pool.cc:224] Handling call
0709 12:11:07.341055 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:07.573213 (+232158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.573253 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:07.573774 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:11:07.770765 (+196991us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:07.771140 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 12:11:07.967192 (+196052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.967304 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.967358 (+    54us) raft_consensus.cc:1685] finished
0709 12:11:07.967374 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.967540 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231724,"prepare.queue_time_us":6854,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":390857,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":10203,"thread_start_us":388713,"threads_started":2,"wal-append.queue_time_us":281005}
I20260709 12:11:07.995747 10334 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 5 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:07.967352 13927 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:08.000118 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 86626.7us
I20260709 12:11:07.968381 14094 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.000694 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.929929 14006 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:08.008492 11178 leader_election.cc:336] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.793s (SENT)
I20260709 12:11:07.966570 13885 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:07.971462 13946 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:07.996619 14083 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:08.020043 14093 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:11:08.005362 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:08.020963 14089 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:11:08.015106 10438 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:07.953356 14087 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:08.033463 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.002039 10334 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:07.961140 14000 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.039690 14000 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.040740 13886 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.042068 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:08.042917 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2191) took 2117 ms. Trace:
I20260709 12:11:08.042074 13948 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.043275 14086 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.044955 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:08.029918 13939 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.043355 10434 rpcz_store.cc:276] 0709 12:11:05.925475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.058228 (+1132753us) service_pool.cc:224] Handling call
0709 12:11:07.058455 (+   227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:07.568405 (+509950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.568441 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:07.568919 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:11:07.775709 (+206790us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:07.775748 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.777109 (+  1361us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:08.042674 (+265565us) raft_consensus.cc:1685] finished
0709 12:11:08.042696 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:08.042841 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":509508,"raft.queue_time_us":513957,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":198018,"threads_started":1,"wal-append.queue_time_us":221860}
I20260709 12:11:08.050333 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.031265 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.052320 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:08.056325 10434 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:08.058362 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:08.058686 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.978268 13949 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.062335 14094 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:08.062363 14087 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:08.064042 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 94733.2us
I20260709 12:11:08.064550 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2193) took 2060 ms. Trace:
I20260709 12:11:08.070865 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.073091 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.047691 13953 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.071322 10431 rpcz_store.cc:276] 0709 12:11:06.004421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.371195 (+1366774us) service_pool.cc:224] Handling call
0709 12:11:07.371659 (+   464us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:07.574760 (+203101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.574794 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:07.575374 (+   580us) log.cc:844] Serialized 52 byte log entry
0709 12:11:07.766670 (+191296us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:07.766707 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.766775 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:08.064334 (+297559us) raft_consensus.cc:1685] finished
0709 12:11:08.064357 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:08.064487 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201207,"raft.queue_time_us":220106,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":67,"thread_start_us":190877,"threads_started":1,"wal-append.queue_time_us":287836}
I20260709 12:11:08.076181 14052 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:08.076363 13948 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:11:08.078351 14103 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:11:08.079108 14094 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:08.080487 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:08.084223 14107 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:11:08.086035 10435 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:08.070070 13978 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.047688 14104 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.091774 13824 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:07.969105 14090 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.086547 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.064607 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:07.994962 14084 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.117199 10437 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:08.117229 10438 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:08.118402 11178 leader_election.cc:400] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9.
I20260709 12:11:08.033679 13927 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:08.090178 14088 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:11:08.000238 13942 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.123837 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.113468 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.124577 10435 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:08.125155 10431 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:07.971805 14091 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.128374 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.119661 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:08.121277 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.129416 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.121626 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.121912 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.122079 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.127017 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:08.127974 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.132171 10511 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:08.133074 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.135073 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2185) took 2225 ms. Trace:
I20260709 12:11:08.132258 11178 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:08.138586 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.137137 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.056783 13801 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.141041 14090 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.115494 13978 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.141304 13801 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.141036 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.142266 10437 raft_consensus.cc:2376] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9. Re-sending same reply.
I20260709 12:11:08.143321 13978 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.143642 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.125831 13952 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:11:08.145632 13952 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:08.145682 13885 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.146166 13952 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:08.143322 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.147506 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.147619 10437 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:08.135538 10432 rpcz_store.cc:276] 0709 12:11:05.909258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.450504 (+541246us) service_pool.cc:224] Handling call
0709 12:11:06.473070 (+ 22566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:07.775119 (+1302049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.775161 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:07.782022 (+  6861us) log.cc:844] Serialized 26 byte log entry
0709 12:11:07.940814 (+158792us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:07.940866 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.940937 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:08.134756 (+193819us) raft_consensus.cc:1685] finished
0709 12:11:08.134784 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:08.134933 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1084016,"raft.queue_time_us":1099057,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":312016,"threads_started":2,"wal-append.queue_time_us":154145}
I20260709 12:11:08.120005 14086 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:08.139385 13967 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:08.149998 13952 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.151337 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.151963 14084 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.152244 10437 raft_consensus.cc:2410] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:11:08.153159 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2187) took 2241 ms. Trace:
I20260709 12:11:08.153249 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:08.153872 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:08.154488 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.155159 10437 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:08.156072 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:08.156682 10437 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.157523 10433 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:08.157657 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.129196 14094 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.163684 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.164069 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.153649 10436 rpcz_store.cc:276] 0709 12:11:05.912286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:07.044743 (+1132457us) service_pool.cc:224] Handling call
0709 12:11:07.045172 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:07.566681 (+521509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:07.566721 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:07.567443 (+   722us) log.cc:844] Serialized 52 byte log entry
0709 12:11:07.716443 (+149000us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:07.716490 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.716562 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:08.089521 (+372959us) spinlock_profiling.cc:243] Waited 9.76 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:08.152785 (+ 63264us) raft_consensus.cc:1685] finished
0709 12:11:08.152814 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:08.153081 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":519732,"raft.queue_time_us":525720,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":22448000,"thread_start_us":148638,"threads_started":1,"wal-append.queue_time_us":266771}
I20260709 12:11:08.161576 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.165601 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.165596 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:08.135545 14099 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:08.170884 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.120244 14104 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.168504 14091 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.164623 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.173651 14083 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.171846 14086 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:08.173794 10432 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:08.174655 14104 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.175082 13949 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.175582 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.176419 10505 raft_consensus.cc:2410] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:11:08.176575 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:08.175155 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.179881 10432 raft_consensus.cc:2410] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:08.171793 13967 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:08.172096 14094 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.173970 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.187690 10429 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 7: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:08.187791 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.148875 13939 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:08.165130 14084 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:08.185053 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.188478 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.160610 13885 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:08.186511 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.188457 10511 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:08.107282 13946 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.202082 10432 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:08.204034 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:08.191105 14084 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.223310 14084 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.226325 13978 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.226936 13978 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.229893 13978 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.194545 13949 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:08.236347 10507 pending_rounds.cc:85] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 5
I20260709 12:11:08.236687 10507 pending_rounds.cc:107] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:11:08.236999 10507 raft_consensus.cc:2889] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:08.237639 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.238546 10436 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:08.212301 14094 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.267561 10432 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:08.221772 13885 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.207448 10437 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:08.279022 14106 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.279194 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.279767 14106 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.283084 14106 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:08.283627 14106 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.284128 14106 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.285773 14101 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.286435 14101 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.288003 10432 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:08.288829 13962 raft_consensus.cc:993] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:11:08.289389 13962 raft_consensus.cc:1081] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:11:08.289639 14006 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.290287 14006 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.291424 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:08.292642 14006 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:08.292697 13953 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:08.293295 13997 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:08.254860 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.294224 14101 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:08.234680 10429 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:11:08.295966 14106 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:08.296546 13927 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.274271 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:08.293356 13953 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:08.299831 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 332878us
I20260709 12:11:08.299482 13953 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.303205 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.303366 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.303954 10585 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:08.304163 10586 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:08.307412 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.308279 10583 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:08.311305 13953 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.320677 10433 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:08.322189 10433 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:08.322400 13997 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:08.323328 13997 consensus_queue.cc:1243] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:11:08.327445 13997 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:08.335630 10436 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:08.244426 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:08.340428 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.341236 10583 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:08.345130 10512 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:11:08.346342 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.347316 10583 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:08.362977 10585 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:08.363158 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:08.384579 10437 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:08.391582 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.364869 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.337615 10430 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:08.394068 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.272176 13924 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.394953 10587 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:08.395291 10583 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:08.342952 13885 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.275158 14033 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.407311 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.417757 10513 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:08.424729 10585 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:08.426235 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.427111 10583 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:08.390388 14096 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:11:08.403386 13924 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.433151 13924 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.415297 13962 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.438603 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.451337 10583 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:08.471951 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.482115 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:08.482822 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 133641us
I20260709 12:11:08.482910 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.463757 14054 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:08.486598 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 510364us
I20260709 12:11:08.486927 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.488107 10584 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:08.489295 10579 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:11:08.505381 10579 pending_rounds.cc:85] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:11:08.505283 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.505715 10579 pending_rounds.cc:107] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.6
I20260709 12:11:08.494184 10583 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:08.506207 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.507154 10584 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:08.507483 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.508270 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.509632 10583 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:08.506014 10579 raft_consensus.cc:2917] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Aborting config change with OpId 7.6: Aborted: Op aborted by new leader
I20260709 12:11:08.511531 10579 raft_consensus.cc:2889] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:08.425916 14033 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:08.518023 14114 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:11:08.518031 13885 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.520546 14054 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.557684 14033 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:08.558799 14108 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.526s	user 0.003s	sys 0.000s
W20260709 12:11:08.607671 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 193014us
I20260709 12:11:08.616472 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.618728 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.621500 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.623828 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.625087 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:08.626435 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.629163 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.633576 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.635510 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.637919 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:08.670217 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:08.705755 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68212.8us
I20260709 12:11:08.714150 14128 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.734304 14128 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.751428 14128 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.756178 13949 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:08.750581 14052 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:08.765666 14054 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.768798 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.770426 13948 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:08.772150 14132 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:11:08.757242 13949 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.786129 13949 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.787910 13959 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.789004 13959 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.793310 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:08.758929 14052 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.795102 14052 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.710330 14137 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 12:11:08.799847 13755 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.805081 13949 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.808617 13949 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.809429 13949 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.810400 13949 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.784860 14134 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:08.813215 13949 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.814661 13959 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:08.820521 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 113548us
I20260709 12:11:08.849241 13755 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.850134 13755 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:08.810889 13952 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.854552 13785 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.854770 13441 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:11:08.855841 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60069.3us
I20260709 12:11:08.868880 13763 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.880570  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:08.869740 13785 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.883692 13984 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.884348 14126 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.884992 14126 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:08.884685 13984 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.885550 14126 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.886114 13785 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:08.891418 13763 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.892755 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.894284 13763 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:08.894556 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.895393 14106 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:08.909783 14127 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 12:11:08.910125 14106 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.898109 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.900502 13962 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:08.902081 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.902520 14134 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:08.904873 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:08.906919 14111 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.846s	user 0.000s	sys 0.005s
I20260709 12:11:08.909083 13755 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:08.910728 14084 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.687s	user 0.020s	sys 0.004s
I20260709 12:11:08.885485 13824 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:08.931387 13885 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.931951 13885 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.932446 13885 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.933190 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.934000 10584 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:08.938536 14084 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.939916 10579 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:08.941162 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.942518 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.939467 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.930964 14132 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:08.946075 13946 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:08.946895 13946 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.947636 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:08.953362 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 95522.8us
I20260709 12:11:08.955500 14135 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:08.955530 13946 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:08.960003 14106 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:08.964648 14000 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:08.956142 14135 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:08.957507 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 452114us
I20260709 12:11:08.954713 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.975309 13962 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:08.976549 14132 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:08.978965 13824 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:08.979386 14143 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 12:11:08.980392 13824 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.983541 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:08.984740 13946 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:08.985277 13924 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.012s
I20260709 12:11:08.985841 13924 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:08.982065 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.987215 10511 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:08.992470 10513 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:11:08.992401 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:08.993726 10584 raft_consensus.cc:2410] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:08.993845 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:08.965816 14000 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:08.999719 10587 raft_consensus.cc:2410] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:11:09.001286 14000 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.003373 10583 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:09.005390 10587 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:11:09.007198 10579 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:11:09.004743 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:09.011349 13953 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:09.011515 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:09.007216 13787 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:11:09.015354 13959 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:09.015596 13984 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:09.016330 14148 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:09.017138 14148 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.019488 10586 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:11:09.010808 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56317.9us
I20260709 12:11:09.022339 10586 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:08.986409 14135 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.047823 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2211) took 1242 ms. Trace:
I20260709 12:11:09.050655 13924 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.054028 13787 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:09.034575 13959 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:11:09.059047 14084 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.048346 10432 rpcz_store.cc:276] 0709 12:11:07.805203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:08.288929 (+483726us) service_pool.cc:224] Handling call
0709 12:11:08.330098 (+ 41169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:08.395836 (+ 65738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:08.395888 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:08.409066 (+ 13178us) log.cc:844] Serialized 26 byte log entry
0709 12:11:08.605493 (+196427us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:08.605663 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:08.605845 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:08.908760 (+302915us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:08.914334 (+  5574us) raft_consensus.cc:1685] finished
0709 12:11:08.914373 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:09.047411 (+133038us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400265cd0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:09.047563 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52857,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9463,"spinlock_wait_cycles":107839744,"thread_start_us":173600,"threads_started":1,"wal-append.queue_time_us":103006}
I20260709 12:11:09.038460 13953 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.079411 14135 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:09.100937 10437 pending_rounds.cc:85] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:09.101397 10437 pending_rounds.cc:107] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:11:09.101718 10437 raft_consensus.cc:2889] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:09.103255 14097 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:11:09.109591 10435 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:11:09.115281 13953 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.122049 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:09.122733 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:09.085124 14128 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.085116 13959 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.139843 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:09.142913 10579 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:09.144426 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:09.146004 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54661.3us
I20260709 12:11:09.140678 10581 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:09.155977 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:09.164839 13953 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.166531 13924 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:09.167387 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.169907 13978 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:09.170647 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:09.170892 13787 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:09.172859 13787 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.170707 13978 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.178782 13978 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.090950 10432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:11:09.199254 10432 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:09.199710 10432 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:09.196934 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.189975 13946 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:11:09.206744 13640 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:11:09.187729 14000 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:11:09.213048 13967 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.213989 13967 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.215778 11221 raft_consensus.cc:2410] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:11:09.188474 14148 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.188488 13959 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.232121 13763 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:09.210543 13953 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:09.239643 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 160034us
I20260709 12:11:09.246644 14128 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:09.249776 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 93401.5us
I20260709 12:11:09.230283 14148 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:09.251214 13967 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.252823 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 514) took 1076 ms. Trace:
I20260709 12:11:09.253531 11221 rpcz_store.cc:276] 0709 12:11:08.176002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:08.176328 (+   326us) service_pool.cc:224] Handling call
0709 12:11:09.252069 (+1075741us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540018fbd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:09.252676 (+   607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2344488832}
I20260709 12:11:09.255833 13885 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.256049 14052 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.258495 13885 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.248404 13953 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.263799 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.264838 14052 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.240054 13787 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.268453 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.291522 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.272280 13967 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:09.301035 13967 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.314350 11212 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:09.354846 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.278478 14126 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.341619 13959 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.385042 10509 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:09.399713 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.409615 10432 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:09.421711 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 116084us
I20260709 12:11:09.422590 10432 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.423648 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:09.424299 10403 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:09.425932 10434 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:09.428604 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:09.407709 13953 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.435212 13967 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.436419 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:09.439180 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.447218 10512 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:09.408594 13885 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:09.417197 14098 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:09.421995 10553 connection.cc:256] RPC call timeout handler was delayed by -1.7288097019998077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
I20260709 12:11:09.462324 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.461910 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:09.484556 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 56094us
I20260709 12:11:09.472828 13959 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:09.484068 13953 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:09.487483 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 189990us
W20260709 12:11:09.491452 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.597s (SENT)
W20260709 12:11:09.502957 14164 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
I20260709 12:11:09.483132 13885 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:09.504670 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:09.507954 14161 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:11:09.510154 14126 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:09.511865 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84863us
I20260709 12:11:09.512858 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.514242 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.519212 13984 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:09.493579 13959 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:09.519882 13763 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:09.536260 13959 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:09.537055 13952 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.584026 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:09.552337 13885 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.555751 13924 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:09.618302 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:09.625627 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:09.649504 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 142753us
I20260709 12:11:09.575942 13952 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:09.653151 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 140458us
I20260709 12:11:09.656348 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:09.677306 13978 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:09.700507 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:09.653303 13885 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.701550 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:09.707060 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 123242us
I20260709 12:11:09.711243 13953 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:09.711942 13953 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.715049 13953 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.715123 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:09.717831 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:09.719185 14170 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:11:09.719805 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.720549 11212 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:09.723415 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.728894 10578 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:11:09.703781 13978 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:09.744419 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.728307 13953 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:09.743914 13967 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:09.787086 13967 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:09.787919 13967 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:09.791504 10553 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.875s (SENT)
I20260709 12:11:09.792152 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:09.799556 14190 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.800361 14190 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:09.801959 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:09.803748 14177 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:11:09.807247 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:09.811158 13942 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.811888 13942 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.812671 13885 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.790731 13967 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:09.750661 13978 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:09.785808 13953 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.822104 13953 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:09.813467 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.819718 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:09.861460 13824 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:11:09.869599 14054 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.870398 14054 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:09.853641 14190 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:09.879495 13952 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:09.924968 13885 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:09.960817 13997 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:09.961052 13885 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:09.974563 14175 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:11:10.021054 10578 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
W20260709 12:11:10.027086 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 309405us
I20260709 12:11:09.993849 13885 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:10.058522 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 276633us
I20260709 12:11:10.060175 14054 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:10.071542 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.064994 14054 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.006s
W20260709 12:11:10.082715 10553 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.898s (SENT)
W20260709 12:11:10.098125 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57013.9us
W20260709 12:11:10.107439 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48477.5us
I20260709 12:11:10.108738 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:10.109737 13885 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:11:10.088140 13967 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:10.119834 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57721.2us
I20260709 12:11:10.118184 14098 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:10.113724 13942 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:10.117352 13967 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:10.134224 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.123035 14098 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:10.135550 13967 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:10.153326 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75192.3us
I20260709 12:11:10.153878 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:10.157845 13885 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:10.168789 13885 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:10.174580 14181 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.465s	user 0.001s	sys 0.000s
I20260709 12:11:10.175539 13885 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.176613 14188 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:11:10.177369 14203 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:11:10.178646 14098 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.208806 14098 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:10.180804 14183 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.393s	user 0.003s	sys 0.000s
W20260709 12:11:10.181473 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3725139590001163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22730000000000003s for a response
W20260709 12:11:10.214615 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2551822450000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 12:11:10.215103 10553 connection.cc:256] RPC call timeout handler was delayed by -1.112982245000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
I20260709 12:11:10.160290 14201 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:10.192710 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 70780.1us
I20260709 12:11:10.219904 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.221767 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.223362 14201 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:10.193720 14135 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:10.180666 13978 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.226222 13978 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:10.162147 13959 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.600s	user 0.006s	sys 0.009s
I20260709 12:11:10.227209 13959 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.229759 13959 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.232228 14135 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:10.233870 14135 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.233166 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.250069 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:10.257457 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106820us
I20260709 12:11:10.266739 13967 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.270474 13967 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:10.276939 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43909.5us
I20260709 12:11:10.256624 14201 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:10.271759 14213 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:10.298768 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.304792 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.308024 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.248898 14148 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:11:10.314674 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.275923 13953 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.329427 13953 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.330080 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.325582 10583 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:10.335673 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.352309 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.355711 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.356844 10586 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:10.358318 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.359579 10583 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:10.300608 13967 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:11:10.365157 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.367444 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.315641 14213 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:10.374434 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:10.368391 14173 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:11:10.396422 10586 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.404042 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.403025 14135 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.406217 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.408121 10581 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:10.408200 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117711us
I20260709 12:11:10.409683 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:10.411278 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 118930us
I20260709 12:11:10.421540 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.425108 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.435685 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.436878 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.443231 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:10.444203 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114270us
I20260709 12:11:10.446769 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.452397 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:10.456446 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.458573 14101 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:10.459216 14101 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.459676 14101 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:10.464439 14186 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:11:10.451736 13953 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.466228 13953 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.468967 13953 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.456748 14213 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.489639 13755 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:10.490522 13755 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.449115 13967 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:10.500170 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45214us
I20260709 12:11:10.513468 10586 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:10.521549 13755 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.523428 14224 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Committing config change with OpId 8.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:10.530368 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.534061 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.539199 10586 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:10.496169 14135 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.488324 14201 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:10.547420 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 88218.5us
I20260709 12:11:10.479926 14101 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.530138 13953 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.541348 13755 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.542274 14121 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:10.560063 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.564924 10509 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:10.566155 14121 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.567534 10581 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:10.567615 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:10.570182 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.571575 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.563580 14201 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.573457 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.566689 14121 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.548465 13967 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.590962 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.563856 14222 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:11:10.597333 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.574962 14201 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.634151 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.635246 10585 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:10.636399 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.569388 10583 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:10.595417 14152 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:10.579561 13953 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.598536 13755 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.657794 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.660312 14233 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:10.661154 14135 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.661104 14233 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.661859 14135 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:10.663414 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98614.1us
I20260709 12:11:10.664224 14135 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.664343 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:10.665969 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:10.666697 11216 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:10.667680 14052 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.668262 14052 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.638052 14201 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.654616 13939 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.694459 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.690531 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.692148 13959 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:11:10.693670 14052 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.706570 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.708091 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.709878 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.712776 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:10.714450 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.714859 14233 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.714696 14188 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:10.715813 14188 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:10.715459 10436 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:10.716900 10509 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:10.717639 10509 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:10.724875 10579 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:10.723778 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:10.726773 11216 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:10.727355 10509 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.689404 14201 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.716251 14188 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.716189 13967 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.735018 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.725214 14101 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:10.777086 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:10.780253 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.781358 10430 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:10.703539 13939 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.805411 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.806604 13755 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.806777 14230 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:10.807668 13953 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.819139 13953 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.819887 13953 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.825142 13755 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.825778 13755 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:10.828022 13755 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:10.831724 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:10.811337 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 79153.2us
I20260709 12:11:10.818176 14237 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.769272 14235 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.835837 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.839632 10505 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:10.806777 13967 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:10.830873 14232 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:11:10.845834 13967 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.848379 13967 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.851966 14201 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.844393 14213 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:10.843128 14231 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 12:11:10.844704 14237 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.854811 13939 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.856474 14237 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.858144 14235 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.860873 14235 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.865139 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.816565 14230 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:10.866146 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:10.871486 10432 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:10.872162 14084 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:10.872668 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.873347 10432 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:10.873903 14121 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.874585 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:10.867489 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.875079 13953 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.887949 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:10.887895 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47283.7us
I20260709 12:11:10.876518 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:10.901275 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.890727 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:10.908154 10585 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:10.911204 10512 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.911849 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:10.912510 10431 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.912997 13997 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:10.913390 13997 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:10.914285 13997 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.915419 13997 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:10.917233 14239 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.917970 14239 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:10.920974 14239 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:10.929909 14121 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:10.906152 13953 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:10.939411 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 275163us
I20260709 12:11:10.939471 14236 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.940680 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.941639 14236 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:10.942173 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2242) took 1555 ms. Trace:
I20260709 12:11:10.942229 14236 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:10.942247 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:10.951171 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 86138.5us
I20260709 12:11:10.958418 10431 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:10.946769 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.964049 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:10.968178 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:10.968051 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.969563 10585 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.967978 10430 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:10.972714 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 82128.2us
I20260709 12:11:10.903328 10436 raft_consensus.cc:2955] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Committing config change with OpId 7.6: config changed from index 3 to 6, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:10.976066 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.986666 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:10.987543 10430 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.946095 14188 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:10.993611 14188 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:10.994020 14189 raft_consensus.cc:1064] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:11:10.995648 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:10.996531 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:10.997529 10431 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:10.942629 10434 rpcz_store.cc:276] 0709 12:11:09.387141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.426923 (+ 39782us) service_pool.cc:224] Handling call
0709 12:11:09.439595 (+ 12672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:09.467312 (+ 27717us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:09.485564 (+ 18252us) log.cc:844] Serialized 14 byte log entry
0709 12:11:10.173900 (+688336us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:10.941121 (+767221us) spinlock_profiling.cc:243] Waited 8.35 ms on lock 0x7b18003468a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c964d73c7 00007f2c92ae99a7 00007f2c92af90bf 00007f2c92af9038
0709 12:11:10.941462 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 12:11:10.941696 (+   234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:10.941759 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:10.941823 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:10.941910 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:10.942071 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14072,"prepare.queue_time_us":1420600,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":11705,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":12662,"spinlock_wait_cycles":55096320,"thread_start_us":14996,"threads_started":1,"wal-append.queue_time_us":182045}
I20260709 12:11:11.002573 10583 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:11.004109 10504 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.943112 14236 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:10.967607 14234 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:11.013833 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71724.2us
I20260709 12:11:11.014315 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.015106 10434 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:11.017695 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.018831 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.019038 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.019580 10432 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:11.020099 10431 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:11.018244 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 225058us
I20260709 12:11:10.994755 13640 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:11.021231 10434 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:11.030406 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.031323 10432 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:11.013500 14234 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.036273 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.037896 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.039096 10431 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:11.042799 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2238) took 1747 ms. Trace:
I20260709 12:11:11.028844 14000 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.024641 13939 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:11.057587 10431 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:11.045607 13885 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:11.039795 14234 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.045224 14235 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.063737 13939 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.073956 14000 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.052268 13824 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.094720 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.098124 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2233) took 1869 ms. Trace:
I20260709 12:11:11.098608 10582 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:11.098659 10435 rpcz_store.cc:276] 0709 12:11:09.229185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.229385 (+   200us) service_pool.cc:224] Handling call
0709 12:11:09.232059 (+  2674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:09.263217 (+ 31158us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:09.395747 (+132530us) log.cc:844] Serialized 14 byte log entry
0709 12:11:09.432283 (+ 36536us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b180037e460. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:09.635340 (+203057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:09.635421 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:09.635492 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:11.091980 (+1456488us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b5400086bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:11.092063 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.098042 (+  5979us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74075,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":65,"raft.queue_time_us":94,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":24046,"spinlock_wait_cycles":89586304,"thread_start_us":318061,"threads_started":2,"wal-append.queue_time_us":396244,"wal-append.run_cpu_time_us":1546,"wal-append.run_wall_time_us":1004604}
W20260709 12:11:11.101774 14249 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:11:11.046871 10433 rpcz_store.cc:276] 0709 12:11:09.295707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.295879 (+   172us) service_pool.cc:224] Handling call
0709 12:11:09.313691 (+ 17812us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:09.380598 (+ 66907us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:09.415262 (+ 34664us) log.cc:844] Serialized 14 byte log entry
0709 12:11:09.535642 (+120380us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:09.545734 (+ 10092us) log.cc:844] Serialized 52 byte log entry
0709 12:11:09.545994 (+   260us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:11.010664 (+1464670us) log.cc:844] Serialized 14 byte log entry
0709 12:11:11.042338 (+ 31674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:11.042426 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:11.042470 (+    44us) raft_consensus.cc:1685] finished
0709 12:11:11.042481 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.042608 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1455560,"prepare.queue_time_us":1462262,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":66,"raft.queue_time_us":66119,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":19181,"spinlock_wait_cycles":6108032,"thread_start_us":152214,"threads_started":3,"wal-append.queue_time_us":208896,"wal-append.run_cpu_time_us":3426,"wal-append.run_wall_time_us":1178914}
I20260709 12:11:11.104481 10433 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:11.108011 13939 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:11.052500 14188 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:11:11.111629 14235 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.115471 10432 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:11.115995 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2240) took 1697 ms. Trace:
I20260709 12:11:11.116413 10429 rpcz_store.cc:276] 0709 12:11:09.346957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.446298 (+ 99341us) service_pool.cc:224] Handling call
0709 12:11:09.446743 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:09.485280 (+ 38537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:09.485317 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:09.536547 (+ 51230us) log.cc:844] Serialized 52 byte log entry
0709 12:11:09.730249 (+193702us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:10.995073 (+1264824us) log.cc:844] Serialized 14 byte log entry
0709 12:11:11.043135 (+ 48062us) log.cc:844] Serialized 14 byte log entry
0709 12:11:11.043442 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:11.043539 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:11.043581 (+    42us) raft_consensus.cc:1685] finished
0709 12:11:11.043591 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.043808 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1231517,"prepare.queue_time_us":1261789,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":11385,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":12712192,"thread_start_us":213318,"threads_started":2,"wal-append.queue_time_us":379094,"wal-append.run_cpu_time_us":2428,"wal-append.run_wall_time_us":1002509}
I20260709 12:11:11.113868 14189 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.121096 10430 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:11.125631 10430 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:11.129129 14000 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:11.089577 13824 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.136323 13824 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.138587 10435 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 12:11:11.139443 14247 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:11:11.140031 13939 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:11.140676 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:11.140609 13939 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.142277 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:11.164134 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 138540us
I20260709 12:11:11.172677 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:11.178108 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.142915 13939 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:11.094192 13640 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:11.182041 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 153799us
I20260709 12:11:11.184401 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:11.188104 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:11.191718 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:11.199924 14236 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.208069 10404 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:11.215039 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 178844us
I20260709 12:11:11.215903 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.212206 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.130462 14230 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.219204 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:11.219300 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.229064 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:11.231308 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:11.232555 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:11.231202 11178 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.092s (SENT) [suppressed 15 similar messages]
I20260709 12:11:11.240764 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.242115 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.243088 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.231732 13997 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:11.244750 13978 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:11.245173 10581 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:11:11.246088 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.246353 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.248803 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:11.232822 14000 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:11.252207 14211 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.252663 14211 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:11.253180 14211 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.252588 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:11.254638 10583 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 5.
I20260709 12:11:11.202863 14236 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:11.260296 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.224102 13967 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:11.263290 14244 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:11.264106 14244 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.264453 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:11.268704 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.270617 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.273530 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.273420 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.276324 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.276883 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.251821 13978 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.279371 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.260372 14220 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:11:11.293080 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.296460 10435 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:11.301381 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.278429 13997 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.304704 13997 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.307071 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.307838 10435 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:11.308905 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.310907 10435 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:11:11.314219 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58208.9us
I20260709 12:11:11.315361 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.316035 10436 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:11.318179 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.277961 14101 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.279596 14222 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:11.279827 13924 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:11.264516 14244 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:11.276445 13942 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.297683 13953 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.335127 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:11.337146 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77234.6us
I20260709 12:11:11.267690 13640 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.340387 13972 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.340946 13972 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.341853 13972 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.344349 14224 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:11.345623 14135 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.346074 14135 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.325158 14233 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:11.346832 14233 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.347299 14233 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:11.347036 14135 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.325836 14222 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.375200 14000 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:11:11.379699 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.381315 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.381808 13824 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.331460 14260 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:11:11.382905 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.386169 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.435735 14250 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:11.438966 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56133.9us
I20260709 12:11:11.439985 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.388753 14084 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.443363 13824 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:11.383203 14236 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.444665 14236 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.445550 13953 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.446152 13953 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.448506 13953 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:11.445154 14236 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:11.449895 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.384723 13978 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.463910 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2243) took 2047 ms. Trace:
I20260709 12:11:11.496433 10585 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:11.496145 10438 rpcz_store.cc:276] 0709 12:11:09.416049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.454821 (+ 38772us) service_pool.cc:224] Handling call
0709 12:11:09.469248 (+ 14427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:09.539284 (+ 70036us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:09.552732 (+ 13448us) log.cc:844] Serialized 14 byte log entry
0709 12:11:10.869794 (+1317062us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b1800346a20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f
0709 12:11:10.871776 (+  1982us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b48000e0358. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f 00007f2c97af6f37 00007f2c97b0b55c 00007f2c97b0b508 00007f2c97b0b488 00007f2c97b0b443 00007f2c97b0a6ec 00007f2c96a34a96
0709 12:11:10.957570 (+ 85794us) log.cc:844] Serialized 14 byte log entry
0709 12:11:11.039366 (+ 81796us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:11.064399 (+ 25033us) log.cc:844] Serialized 52 byte log entry
0709 12:11:11.064576 (+   177us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:11.064833 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 12:11:11.071898 (+  7065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:11.071987 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:11.437299 (+365312us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b1800346a20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:11.463516 (+ 26217us) raft_consensus.cc:1685] finished
0709 12:11:11.463544 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.463712 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2641132,"prepare.queue_time_us":1297031,"prepare.run_cpu_time_us":188,"prepare.run_wall_time_us":12317,"raft.queue_time_us":59088,"raft.run_cpu_time_us":5498,"raft.run_wall_time_us":1338945,"spinlock_wait_cycles":44462336,"thread_start_us":161938,"threads_started":2,"wal-append.queue_time_us":440369,"wal-append.run_cpu_time_us":1460,"wal-append.run_wall_time_us":1005642}
I20260709 12:11:11.336731 13967 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.381309 14101 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.474774 14235 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:11.376766 14222 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:11.502411 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.387458 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.387873 13972 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.394960 14182 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.387738 14234 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:11.385432 13959 raft_consensus.cc:1068] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }.
  Pending config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.400915 14000 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:11.332556 13942 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.373265 13924 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:11.492883 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:11.384547 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.493353 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:11.493777 14250 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.494292 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:11.494864 10586 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:11.495719 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55876.6us
I20260709 12:11:11.503203 10587 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 5.
I20260709 12:11:11.502605 13441 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:11.561692 14219 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:11.562336 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.562904 14230 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.569217 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.570412 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.571393 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.572924 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.574636 10578 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:11.575923 10509 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:11.577111 10513 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:11.578697 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.542613 13942 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.539033 14000 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.582962 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.585654 10580 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:11.587167 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 244569us
I20260709 12:11:11.590721 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:11.474836 14211 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.564013 14148 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 12:11:11.601011 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 265991us
I20260709 12:11:11.601970 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.563705 13924 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.612337 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.617349 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:11.618273 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.622969 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:11.624471 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:11.626663 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.628870 14084 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.628886 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:11.630070 14084 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.630478 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.631342 14084 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.631821 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:11.632695 14084 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.633064 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.634259 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.635191 13755 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.635926 13924 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:11.636011 13755 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:11.635470 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:11.636880 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.637039 13755 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:11.637792 14084 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:11.638435 14084 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:11.637593 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:11.621496 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:11.595189 14101 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.560367 14182 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.580068 13441 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:11.616178 14235 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.647926 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:11.665992 14235 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:11.666472 14235 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:11.667428 10508 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:11.667876 14135 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.597168 14239 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:11:11.673005 14239 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:11.673374 14239 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.675414 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:11.676083 11216 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.622038 13967 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.678876 13967 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.626632 13953 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.655223 14000 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.680245 13953 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:11.661099 14234 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:11.652637 10506 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:11.667424 14235 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.684490 14201 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:11.685010 14201 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:11.685935 14201 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.686374 14234 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:11.687274 14234 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:11.687646 14234 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:11.639786 13885 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.641327 14237 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:11.688747 14234 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:11.659513 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 76692.3us
I20260709 12:11:11.690620 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:11.691851 14250 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.673226 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.693593 10435 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:11.695997 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.698854 10587 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:11.638579 13755 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.704116 14234 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.709851 14250 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:11.710091 14220 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:11.710685 14220 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:11.710489 14250 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.711709 14220 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.696286 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:11.664047 14211 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:11.731431 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:11.733819 10431 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:11.643999 10509 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
W20260709 12:11:11.745313 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54818.3us
I20260709 12:11:11.758474 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:11.760586 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2216) took 1049 ms. Trace:
I20260709 12:11:11.762940 10435 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:11.761248 10509 rpcz_store.cc:276] 0709 12:11:10.693580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:10.701652 (+  8072us) service_pool.cc:224] Handling call
0709 12:11:11.349699 (+648047us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b58007f8bd0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a3b0aa 00007f2c92abd447 00007f2c92ad5fe7 00007f2c92ad77de 00007f2c92ae327c 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c
0709 12:11:11.575419 (+225720us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54002de9a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a54a1d 00007f2c92a3b1a4 00007f2c92abd447 00007f2c92ad5fe7 00007f2c92ad77de 00007f2c92ae327c 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591
0709 12:11:11.742423 (+167004us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646418176}
I20260709 12:11:11.772632 10513 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:11.774758 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.674357 14239 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.774415 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.730533 13441 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:11.787710 13763 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.799733 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.831627 11216 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:11.770753 14148 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:11.851601 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93936.6us
I20260709 12:11:11.853798 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.783288 14000 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:11.789708 14236 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.875844 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:11.871438 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:11.888975 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:11.891669 10334 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 108) took 14413 ms (client timeout 15000 ms). Trace:
I20260709 12:11:11.739282 14084 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.902493 14193 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:11:11.906111 14244 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.001s
I20260709 12:11:11.812976 13763 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:11.924736 10512 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:11:11.929898 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 123078us
W20260709 12:11:11.886077 14233 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.008s
I20260709 12:11:11.933718 10431 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:11.906665 14244 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.938344 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:11.940119 10429 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:11.942207 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 57089.4us
I20260709 12:11:11.943279 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:11.875994 14256 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:11:11.951426 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:11.879484 14148 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:11.958209 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.960706 10510 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:11.892161 10334 rpcz_store.cc:269] 0709 12:10:57.472624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.473109 (+   485us) service_pool.cc:224] Handling call
0709 12:11:11.876465 (+14403356us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:08.526164 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:08.526958 (+   794us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:08.527042 (+    84us) write_op.cc:435] Acquired schema lock
0709 12:11:08.527071 (+    29us) tablet.cc:662] Decoding operations
0709 12:11:08.556657 (+ 29586us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:08.556734 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:08.556759 (+    25us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:11:08.558065 (+  1306us) tablet.cc:701] Row locks acquired
0709 12:11:08.558082 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:11:08.558275 (+   193us) write_op.cc:270] Start()
0709 12:11:08.558450 (+   175us) write_op.cc:276] Timestamp: P: 1783599068558214 usec, L: 0
0709 12:11:08.558481 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:11:08.715313 (+156832us) log.cc:844] Serialized 5116 byte log entry
0709 12:11:09.071310 (+355997us) op_driver.cc:464] REPLICATION: finished
0709 12:11:09.421121 (+349811us) write_op.cc:301] APPLY: starting
0709 12:11:09.421301 (+   180us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:09.421743 (+   442us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:09.423808 (+  2065us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:09.425867 (+  2059us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:09.426299 (+   432us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:11:09.426361 (+    62us) write_op.cc:312] APPLY: finished
0709 12:11:09.426897 (+   536us) log.cc:844] Serialized 95 byte log entry
0709 12:11:09.457015 (+ 30118us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:09.466239 (+  9224us) write_op.cc:454] Released schema lock
0709 12:11:09.466340 (+   101us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:09.475939 (+  9599us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:09.475976 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2539621,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":312431,"apply.run_cpu_time_us":6035,"apply.run_wall_time_us":64956,"num_ops":13,"prepare.queue_time_us":336453,"prepare.run_cpu_time_us":23756,"prepare.run_wall_time_us":245847,"raft.queue_time_us":79355,"raft.run_cpu_time_us":10357,"raft.run_wall_time_us":237923,"replication_time_us":512561,"thread_start_us":472209,"threads_started":4,"wal-append.queue_time_us":148103,"wal-append.run_cpu_time_us":10041,"wal-append.run_wall_time_us":1565274}]]}
I20260709 12:11:11.974756 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:11.971923 10432 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:11.938810 14233 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:11.960011 13763 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.020221 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.009946 14152 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:12.020958 14262 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:11:12.064522 10430 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:12.017557 14267 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 12:11:12.109805 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 153502us
W20260709 12:11:12.109930 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57677.4us
I20260709 12:11:12.110913 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.114703 10437 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:12.063066 13959 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.090726 14152 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:12.165118 14152 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:12.169338 14148 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:12.128662 14244 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.106421 14240 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:11:12.116243 14182 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:12.139092 14268 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:12.139596 14234 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.151701 10429 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:12.116910 13763 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:12.115381 13997 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:11:12.175109 11221 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:11:12.178043 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 319488us
W20260709 12:11:12.183058 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 469561us
W20260709 12:11:12.203248 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.202955 11216 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Committing config change with OpId 8.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:11:12.215559 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.218562 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:12.227890 10585 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:12.234866 10579 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:12.241235 10579 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:12.238525 10583 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:12.198972 14268 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.203694 13997 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:12.222313 14236 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:12.285230 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 173034us
I20260709 12:11:12.242462 10579 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.286903 10579 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:12.221418 14234 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:12.347713 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54681.9us
I20260709 12:11:12.349475 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:12.367578 14228 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.248s
W20260709 12:11:12.371147 14271 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 12:11:12.372912 14280 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 12:11:12.374575 14276 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:11:12.379850 14268 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.386296 14233 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.225579 14148 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 12:11:12.389345 14148 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:12.389885 14148 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:12.391146 14148 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.392904 14148 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:12.285609 13997 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.396319 13997 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:12.407923 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.071s	sys 0.170s
W20260709 12:11:12.424113 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.076s	sys 0.178s
W20260709 12:11:12.424553 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 757437us
I20260709 12:11:12.425987 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:12.404261 14261 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:11:12.398880 14230 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.812s	user 0.019s	sys 0.000s
W20260709 12:11:12.405776 14222 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.905s	user 0.008s	sys 0.009s
I20260709 12:11:12.428529 14228 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:12.443638 10579 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:11:12.448001 10579 pending_rounds.cc:85] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 9
I20260709 12:11:12.448390 10579 pending_rounds.cc:107] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:11:12.448809 10579 raft_consensus.cc:2889] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:12.452770 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:12.453411 10586 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.458696 10584 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:12.360236 14234 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.467120 10579 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:12.469218 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 242658us
I20260709 12:11:12.450923 14222 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.483793 10586 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:12.463100 14228 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.483543 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:12.492385 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:12.493772 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 144419us
W20260709 12:11:12.495291 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 384464us
W20260709 12:11:12.480036 13942 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.890s	user 0.015s	sys 0.000s
I20260709 12:11:12.496771 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.475487 14278 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:12.507356 14285 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:11:12.498632 14291 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.534283 14289 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:12.501503 13942 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.548177 13942 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:12.548810 14228 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.550275 14239 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:12.551622 14201 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.552388 14222 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.555009 14234 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:12.510896 14278 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.562454 14211 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:11:12.570554 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 158005us
W20260709 12:11:12.570765 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 144836us
I20260709 12:11:12.573211 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:12.575649 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 79018.3us
I20260709 12:11:12.576788 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.562950 14211 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:12.578420 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.547984 14294 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:12.581346 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79178.6us
I20260709 12:11:12.579139 14211 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.587488 14234 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.589743 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.590530 10585 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:12.592049 13953 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:12.592511 13953 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:12.592726 13967 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.592872 13953 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.594136 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.595613 11178 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:12.600956 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.602703 14273 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:12.603595 14294 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.604476 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.605553 14273 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.606354 14294 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:12.607190 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.609302 14233 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.609076 14152 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.610471 10578 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:12.611439 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.614380 14233 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:12.616231 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.619853 14233 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:12.562572 14278 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.622823 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:12.624341 14152 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:12.624996 14233 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:12.626338 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.628328 14211 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:12.631477 14233 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:12.631991 14233 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:12.633626 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56965.4us
I20260709 12:11:12.637117 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.637985 10582 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:12.638455 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.640357 14273 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:12.641777 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.642587 14273 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.644011 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.628446 13924 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:12.644948 14273 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.644906 13924 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.646499 14250 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:12.647004 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.647653 13924 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.649631 14273 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.649789 13997 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:12.650957 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.652948 14273 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.654289 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.656232 14273 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.656739 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.657485 10510 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:12.659396 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.661414 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:12.662657 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:12.663316 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:12.666298 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:12.668161 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:12.668994 14054 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.670022 14054 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:12.670431 14054 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:12.671059 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:12.671555 14279 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:12.672013 10437 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:12.670878 14054 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.674592 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:12.675369 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 78520.9us
I20260709 12:11:12.676270 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:12.677270 13942 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:12.677712 13942 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:12.678143 13942 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.680701 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.681493 10586 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:12.684254 10578 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:12.684362 14289 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.634636 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:12.685918 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51400.3us
I20260709 12:11:12.686750 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:12.687726 14250 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:12.688464 14250 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:12.688869 14250 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:12.689254 14250 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.691421 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.695745 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:12.702428 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.707000 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:12.716300 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:12.717586 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.719939 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.721081 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:12.668988 10435 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:12.724220 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.727383 10578 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:11:12.727854 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:12.728487 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.731751 14278 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:12.732218 14278 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:12.732796 14278 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:12.734136 14235 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:12.736778 14220 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:12.737128 14211 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:12.738054 14220 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:12.738242 14211 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:12.738546 14220 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.738737 14211 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.745005 14278 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.746379 14211 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.748404 14211 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:12.749183 14278 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.751943 14220 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.740551 13959 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:12.741014 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:12.759204 14128 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:12.759956 14128 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.761376 14128 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.742409 14213 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:12.743400 14193 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:12.788247 14235 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:12.798082 14235 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.798436 14193 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:12.799013 14193 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.800585 14235 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.800232 14193 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.765748 14245 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:12.802407 14193 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:12.802570 14245 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:12.808040 14245 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:12.809197 14245 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.813432 14245 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:12.796954 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.820768 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.822582 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:12.823392 14235 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:12.823551 10582 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:12.824059 14235 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.827669 13755 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:12.828265 13755 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.831637 13755 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:12.832041 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:12.834333 14235 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:12.834930 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:12.836380 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:12.718611 14101 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:12.718381 14273 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:11:12.837924 14101 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:12.838359 14101 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:12.839215 10333 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 6, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 6 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:12.841130 10333 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 6, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 6 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:12.851308 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:12.729151 14233 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.852029 10580 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:12.852521 10430 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:12.842360 13942 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:12.725836 14152 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:12.857928 13942 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.858059 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:12.860105 10581 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:12.719707 14289 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:12.863375 14289 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:12.863879 14289 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:12.864801 13953 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.719606 13924 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:12.867146 13924 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:12.867540 13924 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:12.871258 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 158696us
I20260709 12:11:12.872347 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.874065 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.874926 14148 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:12.875891 14148 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:12.876396 14148 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:12.876868 14148 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.879383 10508 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:12.880918 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:12.881577 10579 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:12.882102 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.884290 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.885309 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.837007 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:12.889717 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52781.3us
I20260709 12:11:12.842886 10333 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:12.856834 14233 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:12.899971 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 405417us
I20260709 12:11:12.902150 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.886637 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:12.910566 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.910559 10582 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:12.913105 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.916180 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:12.926918 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.927778 10508 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:12.929929 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.931077 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.934345 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.876979 14101 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.935897 10587 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:12.937050 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.939452 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.958549 10512 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:12.941090 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:12.941993 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:12.946197 10579 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:12.879427 13953 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.947573 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:12.969633 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:12.967756 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52122.1us
I20260709 12:11:12.974802 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:12.975366 10505 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:12.975728 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:12.976012 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:12.977106 10583 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:12.981154 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:12.982241 10435 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:12.911911 14273 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:12.985914 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:12.986616 10430 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.986809 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:12.918385 14219 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.988210 14219 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:12.990602 14219 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:12.996438 10513 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:12.997215 10430 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:12.939162 14152 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:12.994215 10333 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 2 to 4, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 4 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:12.957963 14233 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.000902 10430 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:11:12.971326 14234 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:13.004662 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.009119 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:13.006412 10430 pending_rounds.cc:85] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 4
I20260709 12:11:13.010257 10430 pending_rounds.cc:107] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:11:13.010692 10430 raft_consensus.cc:2889] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:13.012066 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:13.012405 10506 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:12.976622 14201 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:13.017269 10429 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:13.021129 14286 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:13.023912 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.983789 14235 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:11:13.028716 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.998294 14273 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.002707 14152 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.047052 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:12.984105 14279 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:11:13.048563 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:13.052783 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:12.983559 14213 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:11:13.057332 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:13.041242 14242 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.031728 14220 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.098403 10579 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:11:13.099390 10587 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:13.044363 14273 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:13.060073 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:13.068347 14180 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:13.107174 14180 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.107649 14180 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:13.068241 13924 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:13.115309 14303 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:11:13.117398 14101 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:13.098920 14242 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.121214 14242 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.127327 10508 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:13.070551 14211 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.129249 14211 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.131805 14211 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.043640 14218 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:13.133448 14218 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:13.134521 14218 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:13.139441 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.139178 14250 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.141579 14250 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:13.142112 13972 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.142669 13972 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.144850 13972 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:13.062397 14084 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:13.151515 13924 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.155556 10581 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:13.158100 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:13.160775 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.160532 10333 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 4 to 5, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:13.062860 13967 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.164908 14289 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.061932 14201 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 12:11:13.166249 14201 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:13.166613 14201 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:13.167523 14201 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.168815 14201 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:13.170389 13959 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:13.056083 13953 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:11:13.171537 14202 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:13.171958 14202 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:13.073045 14213 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:13.172667 14213 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:13.172760 14202 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.173153 14180 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.173625 14213 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.024744 14266 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.073143 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:13.012202 14233 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.175421 10504 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.176905 14233 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.177270 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.177733 10429 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:13.177888 10585 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:13.073979 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:13.075192 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:13.179466 10580 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:13.075682 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:13.179867 14299 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.180435 14299 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.181541 14233 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.182106 14233 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.059803 14152 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:13.025032 14228 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:11:13.105825 13942 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:13.184682 13942 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:13.185699 13942 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.186688 10583 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:12.991283 14054 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.076138 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:13.186920 14266 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:11:13.188217 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 128246us
I20260709 12:11:13.162344 10333 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 7, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:13.194315 10585 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:13.195472 10583 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:13.182650 10580 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:11:13.208402 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 133766us
I20260709 12:11:13.216486 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:13.206615 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:13.220019 10580 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.224000 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:13.224119 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.233984 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.224921 10435 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:13.184595 14234 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.239338 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:13.240245 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:13.249284 10507 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.241736 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:13.250248 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57964.5us
I20260709 12:11:13.251803 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:13.247576 10333 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 2 to 5, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 5 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:13.191128 14213 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:13.262576 10580 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:13.269300 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:13.245410 10585 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:13.142171 14250 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.277526 10582 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:13.277105 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:13.277226 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.228727 13967 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:13.221167 13978 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.313555 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:13.281605 10433 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:13.228513 14266 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:11:13.307065 14054 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.308413 14299 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:13.309464 14202 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:13.329622 11217 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:11:13.311023 13972 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:13.311687 14308 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.318596 10504 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:13.277669 13755 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:13.231765 13787 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:11:13.352135 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:13.301082 13967 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.331965 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:13.338945 11216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 541) took 1142 ms. Trace:
I20260709 12:11:13.358489 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:13.338181 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 397207us
W20260709 12:11:13.345112 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46889.7us
I20260709 12:11:13.347404 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:13.339931 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.364073 10434 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:13.293196 14235 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.365550 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.298087 14250 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.270219 13939 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:11:13.362534 11216 rpcz_store.cc:276] 0709 12:11:12.190376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.190513 (+   137us) service_pool.cc:224] Handling call
0709 12:11:12.202552 (+ 12039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:12.202779 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:12.418675 (+215896us) log.cc:844] Serialized 14 byte log entry
0709 12:11:13.324234 (+905559us) log.cc:844] Serialized 14 byte log entry
0709 12:11:13.330865 (+  6631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:13.330932 (+    67us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:13.331013 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.331926 (+   913us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:13.332435 (+   509us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3629,"raft.queue_time_us":16804,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":14658,"thread_start_us":731698,"threads_started":3,"wal-append.queue_time_us":106473}
I20260709 12:11:13.328662 14289 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:13.373693 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:13.397827 10436 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:13.330014 14193 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:11:13.374724 11217 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:13.328353 14180 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:13.335397 14148 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.335790 14101 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.356137 11218 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:13.382125 10509 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:13.338255 13972 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.387809 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:13.418648 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55033.8us
I20260709 12:11:13.374414 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:13.369706 13967 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:13.367527 14260 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:13.294499 14211 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:11:13.402362 10507 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:13.375375 13924 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:13.375265 14084 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.375066 13978 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:13.377794 14128 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.387051 13959 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:11:13.408660 10584 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:13.411459 11218 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:13.420264 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:13.395583 14291 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:13.462848 14291 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:13.464077 14291 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:13.467692 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47546.4us
I20260709 12:11:13.467886 13959 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:13.468431 13959 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:13.378232 14235 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.469416 13959 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.470674 14235 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.439917 10581 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:13.471556 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.455958 10333 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 4 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 7 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:13.424245 13972 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:13.425307 14242 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:13.446893 10436 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:13.491685 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:13.497871 11221 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:13.498368 14311 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.499173 14311 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.502151 14311 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:13.502800 14277 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.503751 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:13.492528 10553 connection.cc:256] RPC call timeout handler was delayed by -2.5735941430000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
W20260709 12:11:13.505606 10553 connection.cc:256] RPC call timeout handler was delayed by -2.4736941430001025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
W20260709 12:11:13.506042 10553 connection.cc:256] RPC call timeout handler was delayed by -2.1298941430000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 12:11:13.490101 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 213131us
I20260709 12:11:13.508237 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:13.508490 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.492599 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:13.457695 14270 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:13.518296 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.450965 13924 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:13.435923 14260 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.492844 13953 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.550336 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:13.527668 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:13.549744 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:13.561012 14310 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:11:13.560091 14312 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:11:13.451102 14084 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.456422 13978 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.472669 14128 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.485461 14266 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:13.594846 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67294.7us
I20260709 12:11:13.489810 13959 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:13.688133 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:13.680156 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.680835 10580 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:11:13.699052 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.700589 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.701939 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.681059 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.592218 10579 raft_consensus.cc:606] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:13.704315 10579 raft_consensus.cc:614] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:13.692906 11178 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:13.683035 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:13.705788 10583 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:13.714673 10438 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:13.719499 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:13.721778 14311 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:13.722353 13939 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:13.722385 14311 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.723344 13939 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:13.724674 13939 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.571205 14270 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:13.726450 14311 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:13.727240 14270 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.595603 14235 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:13.730042 14235 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.594668 13953 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.732962 14235 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:13.734318 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:13.719693 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.663535 14266 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.736059 10432 raft_consensus.cc:2368] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:11:13.737848 14266 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.608397 14148 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:13.643882 14101 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:13.713515 10478 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:13.620913 14128 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.746279 14218 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:13.746712 14218 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:13.747824 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 61704.2us
I20260709 12:11:13.747094 14218 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:13.777742 14307 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.359s	user 0.000s	sys 0.002s
I20260709 12:11:13.779188 14148 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.779918 14148 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.782897 14148 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:13.646912 14316 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:11:13.593168 14308 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:13.704403 10587 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:13.568084 14250 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:13.849381 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 101691us
I20260709 12:11:13.851089 10478 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:13.851338 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2276) took 1031 ms. Trace:
I20260709 12:11:13.851882 10578 rpcz_store.cc:276] 0709 12:11:12.819803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.877797 (+ 57994us) service_pool.cc:224] Handling call
0709 12:11:12.878215 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:12.878938 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:12.878969 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:12.914429 (+ 35460us) log.cc:844] Serialized 26 byte log entry
0709 12:11:13.328929 (+414500us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:11:13.328978 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.329061 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:13.646029 (+316968us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:13.850907 (+204878us) raft_consensus.cc:1685] finished
0709 12:11:13.850941 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:13.851252 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":20896000,"thread_start_us":328990,"threads_started":1,"wal-append.queue_time_us":480045}
I20260709 12:11:13.854146 14298 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:13.610810 14084 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:13.613448 13978 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:13.855633 13978 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:13.856110 13978 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.856544 13978 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.871335 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:13.713874 13755 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:13.882329 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:13.489521 14314 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:13.885442 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:13.890868 14224 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:13.896862 14250 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.627458 13924 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.897962 13755 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.877573 14101 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:13.842644 14308 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:13.910838 14201 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.007s
I20260709 12:11:13.566975 14273 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:13.911542 11212 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:11:13.912791 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 492958us
W20260709 12:11:13.920418 10510 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.002s
I20260709 12:11:13.961321 10510 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:13.962185 10433 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:13.963585 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:13.920826 10512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.053s	sys 0.064s
W20260709 12:11:13.925595 10431 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.007s	sys 0.003s
W20260709 12:11:13.964334 10512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.054s	sys 0.064s
W20260709 12:11:13.964411 10431 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.008s	sys 0.003s
I20260709 12:11:13.965451 14302 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:13.965879 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:13.966498 10431 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:13.967373 14128 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:13.927629 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.968631 10435 raft_consensus.cc:2368] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:11:13.935098 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.969898 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:13.970451 10436 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.970875 14219 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:13.971930 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:13.972878 10435 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:13.953188 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.978997 10436 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:13.959444 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.903748 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.981257 10438 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:11:13.981109 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:13.522673 14242 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:13.992671 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:13.994387 10434 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:11:13.995739 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:13.997270 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.000618 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.004822 10433 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.984613 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.061769 10438 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:14.062469 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.007207 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.012126 13755 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.013931 10430 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:13.935994 14314 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:14.018699 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:14.027814 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:14.030249 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.066612 10431 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:14.033303 10587 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:13.933145 14250 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.048913 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:14.021744 14152 consensus_meta.cc:287] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.833s	user 0.013s	sys 0.001s
I20260709 12:11:14.054623 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.069218 10429 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:14.067245 10437 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:14.070932 10435 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:14.057222 10582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.053s	sys 0.101s
W20260709 12:11:14.057933 10586 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.005s
W20260709 12:11:14.050679 14213 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.772s	user 0.007s	sys 0.006s
I20260709 12:11:14.060735 10436 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:14.055397 14245 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:14.073019 10586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.007s	sys 0.006s
W20260709 12:11:14.072763 10582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.053s	sys 0.101s
W20260709 12:11:14.070616 13967 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.631s	user 0.013s	sys 0.000s
I20260709 12:11:14.091954 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:14.063391 14291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.042s
I20260709 12:11:14.093525 13967 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.097589 10436 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:14.101536 14260 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:14.102286 14233 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.113904 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1861) took 1405 ms. Trace:
W20260709 12:11:14.156107 10333 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.028s	sys 0.038s
I20260709 12:11:14.132274 13946 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:14.156386 10510 rpcz_store.cc:276] 0709 12:11:12.708201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.708429 (+   228us) service_pool.cc:224] Handling call
0709 12:11:14.113682 (+1405253us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54000d9b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:14.113803 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097053056}
I20260709 12:11:14.149883 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.150091 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:14.088979 14302 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:14.134807 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.079900 14152 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:14.157028 10333 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.028s	sys 0.038s
I20260709 12:11:14.062309 14308 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.164916 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.152105 14268 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:14.187497 10432 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:14.087814 14245 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:14.152457 14291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.008s	sys 0.043s
I20260709 12:11:14.194392 10435 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:14.200323 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.209704 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:14.210750 10583 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:14.211109 10585 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:14.212116 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.212776 10435 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:14.215098 10510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:14.220563 11221 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:14.202165 14233 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.236395 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.239120 10435 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:14.186877 14302 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.221477 14273 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.252594 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2309) took 1187 ms. Trace:
I20260709 12:11:14.220384 14219 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:11:14.239346 14233 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.287055 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.247429 14286 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:14.283880 10430 rpcz_store.cc:276] 0709 12:11:13.065219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.068172 (+  2953us) service_pool.cc:224] Handling call
0709 12:11:13.068487 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.065069 (+996582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.252319 (+187250us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540015f9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:14.252525 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173716352}
W20260709 12:11:14.254884 14323 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:11:14.261072 14152 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:14.267048 14302 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.286012 14311 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:14.256791 13953 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:14.291347 14286 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:14.262348 13967 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.267741 14250 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:14.292665 13972 consensus_meta.cc:287] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.787s	user 0.014s	sys 0.000s
I20260709 12:11:14.259855 14245 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:14.302589 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 319470us
I20260709 12:11:14.350777 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:14.351053 10433 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:14.261754 13978 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.337378 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.353229 10436 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:14.337466 10584 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:14.211054 14314 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.340266 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:14.354497 13978 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.340943 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.309973 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:14.355540 14228 raft_consensus.cc:2764] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:14.355854 11221 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:14.282146 14101 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 12:11:14.367865 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57986.1us
I20260709 12:11:14.369791 10552 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:14.379410 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:14.388193 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:14.390908 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2250) took 1169 ms. Trace:
I20260709 12:11:14.403983 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:14.372421 14218 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.006s
W20260709 12:11:14.370456 14202 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.034s	user 0.015s	sys 0.000s
W20260709 12:11:14.407881 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 184399us
I20260709 12:11:14.415633 10553 leader_election.cc:400] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:11:14.416145 10553 leader_election.cc:403] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:14.418497 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:14.419901 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:14.420878 10581 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:14.422276 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:14.434511 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:14.435634 10584 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:14.392513 10512 rpcz_store.cc:276] 0709 12:11:13.213110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.213288 (+   178us) service_pool.cc:224] Handling call
0709 12:11:13.213645 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:13.248283 (+ 34638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:13.248319 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:13.254939 (+  6620us) log.cc:844] Serialized 26 byte log entry
0709 12:11:13.964844 (+709905us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:11:13.964888 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.964961 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:14.382183 (+417222us) raft_consensus.cc:1685] finished
0709 12:11:14.382219 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:14.382395 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62969,"raft.queue_time_us":35917,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":9330,"thread_start_us":643111,"threads_started":1,"wal-append.queue_time_us":748459}
I20260709 12:11:14.389107 14250 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:14.461828 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:14.462049 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.467937 10436 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:14.463435 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:14.450098 10511 raft_consensus.cc:2410] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:11:14.469175 10584 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:14.502408 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 132703us
I20260709 12:11:14.406463 14152 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.370927 13972 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:14.538111 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71478.4us
I20260709 12:11:14.602335 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:14.379393 14302 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
W20260709 12:11:14.540167 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 124758us
W20260709 12:11:14.420135 13939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.019s	sys 0.018s
W20260709 12:11:14.542371 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 130182us
I20260709 12:11:14.547173 14273 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:14.547576 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 136096us
I20260709 12:11:14.566967 14201 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.476629 13924 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.599436 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:14.599697 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:14.544606 14101 consensus_queue.cc:1243] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:14.538772 13967 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.541885 13953 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.665046 13953 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.665493 14345 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.666033 14345 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.667402 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.668473 14345 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:14.669065 14345 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:14.424739 14218 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.673060 14218 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.547402 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:14.675032 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:14.675902 14340 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:11:14.677363 14336 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.329s	user 0.000s	sys 0.000s
I20260709 12:11:14.678346 13953 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:14.678900 14332 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:11:14.679179 13953 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.683320 10505 raft_consensus.cc:2368] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
I20260709 12:11:14.683084 13953 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.687168 13953 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:14.540766 13959 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.825s	user 0.014s	sys 0.000s
I20260709 12:11:14.698102 10510 raft_consensus.cc:606] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:14.698529 10510 raft_consensus.cc:614] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:14.613426 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:14.617434 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.572724 14242 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:14.607826 13939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.026s	sys 0.027s
W20260709 12:11:14.660652 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 52274.7us
I20260709 12:11:14.653859 10506 raft_consensus.cc:2368] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:14.572135 13755 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.659742 10509 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:14.660357 10504 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
W20260709 12:11:14.662801 14335 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:11:14.653097 14268 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Committing config change with OpId 9.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:14.616611 14273 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.618857 14250 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:14.663374 14233 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.663684 14343 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:11:14.661823 14152 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.606950 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:14.696986 13972 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.694731 13967 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.711983 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:14.714385 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.760771 10587 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:14.715561 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:14.762382 10580 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:14.712042 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:14.546399 14286 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.654901 14201 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.721257 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.767721 10585 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:14.707204 14224 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.722170 13953 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:14.660373 13924 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.722415 13755 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.724252 11212 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:14.739339 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.746208 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.738075 13939 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.743675 14233 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.759841 14250 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:14.734226 14203 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:11:14.807356 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 201515us
I20260709 12:11:14.812258 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:14.817682 10580 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:14.805886 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:14.818801 10579 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:14.804019 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 201778us
I20260709 12:11:14.796489 14239 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:14.825711 10432 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:11:14.797257 14348 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:14.788025 14291 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:11:14.779412 14224 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.839591 10585 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:14.804211 14233 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.847100 14334 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.831471 10580 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:14.790336 13953 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.815255 14314 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.853328 10513 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:11:14.855100 10513 pending_rounds.cc:85] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 9
I20260709 12:11:14.855470 10513 pending_rounds.cc:107] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:11:14.855778 10513 raft_consensus.cc:2889] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:14.856823 10511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 365) took 1783 ms (client timeout 2173 ms). Trace:
I20260709 12:11:14.865505 13953 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:14.862505 10511 rpcz_store.cc:269] 0709 12:11:13.073015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.073703 (+   688us) service_pool.cc:224] Handling call
0709 12:11:14.852677 (+1778974us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002c65d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:14.856746 (+  4069us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2907766144}
I20260709 12:11:14.830000 14239 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:14.802799 13939 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.835645 14349 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:14.875505 14341 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:11:14.875778 14201 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:14.875973 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:14.924944 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114928us
I20260709 12:11:14.876855 14273 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.877404 14250 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.867498 13967 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:14.875808 13953 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.881091 14152 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:14.843796 14101 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:14.882548 13924 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:14.888761 14239 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.893483 14311 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.895147 10582 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:14.955238 10582 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:14.896425 14325 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:14.898545 13755 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:14.915961 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.921852 10508 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:14.926718 10585 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:14.927471 10579 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:14.964179 10585 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:14.923965 14203 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.907054 14286 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:14.960659 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.961496 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:14.951408 10553 leader_election.cc:400] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:11:14.971331 10553 leader_election.cc:403] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:14.870597 14314 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.867898 14334 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.970886 11216 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:14.967496 10582 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.946535 14273 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:14.956784 14325 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:14.966351 10585 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:14.957350 13967 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.008543 10585 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:14.961681 13924 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.957508 14311 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:14.989926 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:14.924373 14349 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:14.969521 14337 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.983191 14239 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:14.948652 13953 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:15.050774 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 108344us
I20260709 12:11:15.023810 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:15.037338 13755 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.039069 14308 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:15.041345 14353 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:11:15.060686 13755 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.062017 14314 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:15.003362 14152 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7.
I20260709 12:11:15.064090 13755 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.064234 14334 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.013602 14242 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.056707 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.011127 14219 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:15.080232 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:15.060904 14348 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.091425 13755 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.018298 13924 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.066334 14218 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:15.093909 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:14.983951 14203 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.094705 10432 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:15.010726 14273 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.018146 13967 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:15.110306 10434 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:15.125327 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:15.100805 10430 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:15.126917 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.127665 10437 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:15.133195 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.133765 13640 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:15.133958 10437 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
W20260709 12:11:15.135954 14354 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:11:15.127333 14362 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:11:15.140305 14362 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:15.140712 14362 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:15.141916 14362 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.094065 13755 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.146266 13755 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:15.135874 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 84036.5us
I20260709 12:11:15.046444 14337 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.152410 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.153905 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:15.103294 10435 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:11:15.159729 13755 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.077018 14260 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:11:15.168807 10435 pending_rounds.cc:85] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 7
I20260709 12:11:15.092054 14356 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:15.177904 10435 pending_rounds.cc:107] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:11:15.173868 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:15.175166 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 274273us
I20260709 12:11:15.178493 10435 raft_consensus.cc:2889] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:15.084695 14219 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.083031 14242 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.189968 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:15.199384 10430 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:15.192981 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:15.195969 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 363142us
I20260709 12:11:15.202446 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.204453 14193 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:15.197635 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:15.206697 10435 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.203164 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.205472 14314 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:15.175442 14152 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:15.212432 14193 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.180389 13959 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:15.179262 14356 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.222283 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.224731 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.234215 10430 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:15.239810 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.095470 14348 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.244544 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.179538 13755 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.084326 14325 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.213973 14349 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.269831 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:15.273188 10435 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:15.274322 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:15.275032 10437 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.215597 14351 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:15.222594 14356 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:15.273257 14367 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:11:15.249579 14270 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:15.281374 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:15.284590 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.244293 13939 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:15.255986 14347 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:15.243340 14365 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:15.289402 14360 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 12:11:15.287595 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 85234.6us
I20260709 12:11:15.303503 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:15.304522 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.304374 10582 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:15.305429 10506 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2252) took 1953 ms (client timeout 2021 ms). Trace:
I20260709 12:11:15.306005 14337 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.306874 13924 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.307332 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1865) took 1317 ms. Trace:
I20260709 12:11:15.311779 11218 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:11:15.310467 10506 rpcz_store.cc:269] 0709 12:11:13.351620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.367398 (+ 15778us) service_pool.cc:224] Handling call
0709 12:11:15.305192 (+1937794us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002292d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:15.305354 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2759046528}
I20260709 12:11:15.315482 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.276249 14218 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:15.237507 14193 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.286522 14201 raft_consensus.cc:2764] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:15.273928 14282 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:15.315264 10505 rpcz_store.cc:276] 0709 12:11:13.989493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.989651 (+   158us) service_pool.cc:224] Handling call
0709 12:11:15.307092 (+1317441us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54003fded4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:15.307255 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586001152}
I20260709 12:11:15.269019 13959 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 12:11:15.353953 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 109502us
I20260709 12:11:15.355805 11216 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
W20260709 12:11:15.364540 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.366431 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:15.367399 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.307305 13953 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.310400 14348 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.362957 14218 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.352032 14351 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:15.386166 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 594) took 1106 ms. Trace:
I20260709 12:11:15.386736 14286 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:11:15.387384 14342 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:11:15.411262 14203 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.411697 14311 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.396433 13755 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.400000 14193 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:15.403821 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 93980us
W20260709 12:11:15.372336 14370 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:11:15.423547 14242 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.426187 13967 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:15.406112 10437 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:15.373008 14344 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:11:15.413199 14268 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.578s	user 0.015s	sys 0.000s
I20260709 12:11:15.399243 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.421151 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.407167 10582 rpcz_store.cc:276] 0709 12:11:14.280103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.280261 (+   158us) service_pool.cc:224] Handling call
0709 12:11:14.280575 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.895013 (+614438us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540042cd20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae11c0 00007f2c92ae2231 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c
0709 12:11:15.304112 (+409099us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b58005f2dd0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a3b0aa 00007f2c92abd447 00007f2c92ad5fe7 00007f2c92ad77de 00007f2c92ae1b49 00007f2c92ae2416 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4
0709 12:11:15.382209 (+ 78097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.382240 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.385780 (+  3540us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:15.385839 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.385928 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.386071 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10003,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":4218,"spinlock_wait_cycles":1473912064}
I20260709 12:11:15.407155 14218 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:15.443279 14219 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.436206 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.445986 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.437537 14351 raft_consensus.cc:2764] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:11:15.418594 14314 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:15.439306 14328 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.036s	user 0.000s	sys 0.007s
I20260709 12:11:15.456759 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.459221 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.451148 14345 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:15.461488 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.457125 14152 raft_consensus.cc:2764] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:11:15.461882 14343 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:15.459769 14351 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:15.460763 14233 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.471083 14300 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:15.485160 14300 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:15.486965 14300 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.488543 14300 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.491428 14343 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:15.492587 14343 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.497179 14311 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.472360 14345 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.503573 14345 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:15.516079 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54665us
W20260709 12:11:15.517354 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.476239 14314 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:15.541394 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:15.537719 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76172.5us
I20260709 12:11:15.521054 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.546506 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2280) took 2661 ms. Trace:
W20260709 12:11:15.546976 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 280245us
I20260709 12:11:15.548877 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:15.549613 14374 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:11:15.548491 10586 rpcz_store.cc:276] 0709 12:11:12.883514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.887158 (+  3644us) service_pool.cc:224] Handling call
0709 12:11:12.935085 (+ 47927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:13.422618 (+487533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:13.422655 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:13.423258 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 12:11:14.082413 (+659155us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:14.082496 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.082595 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:15.438537 (+1355942us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:15.542422 (+103885us) raft_consensus.cc:1685] finished
0709 12:11:15.542461 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.544834 (+  2373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4855,"raft.queue_time_us":182628,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":9617664,"thread_start_us":1016234,"threads_started":2,"wal-append.queue_time_us":808624}
I20260709 12:11:15.554061 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.527330 14152 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:15.522074 13939 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.556121 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.552039 14218 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.560451 14245 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:15.564060 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.565290 14387 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.565658 14224 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:15.561522 14382 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
I20260709 12:11:15.565945 14387 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.568604 14387 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.570313 14350 raft_consensus.cc:2764] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:15.556027 14379 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:15.572063 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 287561us
I20260709 12:11:15.557390 14101 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:15.574009 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.581185 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:15.573549 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:15.583171 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:15.571467 14245 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.563352 14152 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:15.587939 14365 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:15.591853 14376 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:15.581698 14224 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:15.587821 14268 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:15.574559 14379 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.593844 14218 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.563936 13939 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.614656 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:15.615012 14352 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.615453 11221 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.615736 14352 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.617208 13939 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:15.607980 14376 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:15.608331 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:15.620301 11218 raft_consensus.cc:1813] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Handling vote request from an unknown peer 33d77e3063684586a737bccf358786d8
I20260709 12:11:15.620227 14376 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.622059 11218 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:15.590667 14245 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:15.590287 14101 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:15.613286 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.571151 14219 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:11:15.614372 11217 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:15.625252 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44172.2us
I20260709 12:11:15.628679 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.627915 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.631891 10586 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:15.633179 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.633536 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.602519 14233 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.637739 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.635537 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.640074 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.638481 10581 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:15.644618 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.645167 10586 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:15.645238 10580 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 6.
I20260709 12:11:15.638806 14308 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:11:15.641888 14233 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.654294 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.659428 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.660056 10587 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:15.660396 11221 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:15.625464 14379 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:15.629769 14268 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:15.626430 14224 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:15.663618 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.666944 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.667300 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.668238 10433 raft_consensus.cc:1813] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Handling vote request from an unknown peer 33d77e3063684586a737bccf358786d8
I20260709 12:11:15.668666 10579 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:15.668857 10433 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:15.669243 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.669883 10587 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:15.670205 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:15.670742 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.667313 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.674057 10438 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:15.674340 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.669709 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.675477 14393 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:15.676633 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.676983 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.677408 10429 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:15.678028 10508 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:11:15.679931 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.680415 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.680812 10508 raft_consensus.cc:2410] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:11:15.681718 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.682634 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 463) took 1441 ms. Trace:
I20260709 12:11:15.682426 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:15.683081 11212 rpcz_store.cc:276] 0709 12:11:14.241532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.241738 (+   206us) service_pool.cc:224] Handling call
0709 12:11:14.258840 (+ 17102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.789494 (+530654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.682385 (+892891us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400459054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:15.682566 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069649664}
I20260709 12:11:15.684242 10430 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:15.684758 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.678583 14391 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:15.682351 10505 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:15.687911 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.667639 14233 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.675928 14393 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.691408 14393 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:15.693614 14352 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.694509 14224 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.697300 14356 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.699249 14391 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:15.700275 14379 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.703387 13755 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:15.703672 14219 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:11:15.703715 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 90534.1us
I20260709 12:11:15.706419 14219 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:15.707327 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.708532 14348 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:15.707356 14219 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.710171 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:15.704737 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56738.4us
I20260709 12:11:15.714522 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.715443 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.716835 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.686833 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.719148 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.720345 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.722337 14379 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:15.719880 10438 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:15.725489 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.726426 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.731705 10403 leader_election.cc:400] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:15.731977 10403 leader_election.cc:403] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:15.735477 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.737053 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.724226 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.741317 10433 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:15.751938 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.752090 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.752395 11221 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:11:15.752892 10432 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:15.753544 14291 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.754215 13924 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:15.754086 14291 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.754639 13924 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.755045 13924 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:15.755918 14393 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.756601 14291 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.757140 10434 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:15.759294 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.759860 14273 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.761598 14270 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:15.762282 14270 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.762949 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.763408 14268 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.764000 10580 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:15.764137 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.764472 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2363) took 1053 ms. Trace:
I20260709 12:11:15.762751 14270 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.764886 10583 rpcz_store.cc:276] 0709 12:11:14.711227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.711561 (+   334us) service_pool.cc:224] Handling call
0709 12:11:15.763717 (+1052156us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:11:15.763914 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:15.764106 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.764132 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.764206 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:15.764238 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.764295 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.764410 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13348096}
I20260709 12:11:15.766186 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:15.766640 14270 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.766798 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.767956 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.752310 14361 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:15.770239 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:15.770819 10438 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:15.771660 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:15.772439 10583 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:15.772506 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:15.776127 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.777733 10579 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:15.752846 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.779228 10513 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:15.779381 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:15.727792 10429 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:15.784619 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2362) took 1056 ms. Trace:
I20260709 12:11:15.786280 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:15.786360 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.785035 10584 rpcz_store.cc:276] 0709 12:11:14.710854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.711099 (+   245us) service_pool.cc:224] Handling call
0709 12:11:15.766633 (+1055534us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:11:15.767592 (+   959us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2746112}
I20260709 12:11:15.790345 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.791128 10508 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:15.792709 10510 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:15.795439 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.703843 13755 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.798708 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.725271 14352 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:15.803018 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.803545 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.804548 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.805845 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.801848 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:15.810248 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.811457 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.784844 14322 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:15.810848 10506 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:15.770143 14361 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.784066 14350 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:15.814491 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:15.830646 10505 raft_consensus.cc:2376] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10. Re-sending same reply.
I20260709 12:11:15.823936 13997 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:15.832472 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.801702 13755 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:15.806246 14300 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:15.802649 14352 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:15.796526 14233 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:11:15.868929 14356 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.870136 14393 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:15.837633 14361 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:15.872752 14300 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:15.874720 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:15.826426 14322 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:15.880465 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:15.887557 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65780.1us
I20260709 12:11:15.890717 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.891626 10583 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:15.892905 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.898941 10479 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.880707 14352 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:15.898357 11218 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index 2 to 7, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:15.916111 10583 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:15.922621 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.915997 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.700361 14391 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.920050 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:15.922318 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:15.933090 10430 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:15.897480 14322 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:15.936980 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.918219 13640 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.882941 14345 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.952286 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:15.922052 14375 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:15.953624 14375 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.954010 14375 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:15.963002 14375 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.966296 14375 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:15.967940 14404 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:11:15.969206 14397 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:11:15.971341 14395 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:11:15.938016 14393 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:15.972491 14393 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:15.973552 14393 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:15.980417 14300 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:15.948510 14193 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:15.982651 14193 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.983067 14193 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:15.985204 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:15.950521 14387 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:15.986502 14387 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.986887 14387 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:15.987346 14359 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:15.987792 14359 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.988157 14359 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:15.999625 14407 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:11:16.004249 10510 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:16.010967 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58557.3us
I20260709 12:11:16.016999 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:16.018134 10578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:16.022719 14392 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.023602 14345 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.023681 14392 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.026437 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:16.030787 14345 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.031538 14345 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.054358 10438 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:16.064020 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:16.065608 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48677.8us
I20260709 12:11:16.067842 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:16.068987 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:16.071264 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58400.4us
I20260709 12:11:16.088119 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:16.089071 14375 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:16.089622 14375 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:16.090564 14375 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.098287 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:16.100389 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:16.098340 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:16.085680 14101 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:16.032784 14152 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.149457 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:16.042964 14314 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.008265 13640 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.033222 14379 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.023543 13755 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.146580 14101 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:16.144105 14343 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:16.178409 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:16.181070 10506 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:16.167167 14416 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:16.173094 10510 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:16.189414 13755 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.192837 10435 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:16.148736 14392 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.160847 14298 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:16.206070 14423 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.208525 10552 leader_election.cc:400] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
I20260709 12:11:16.217622 10552 leader_election.cc:403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:16.179929 14343 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.158089 14152 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.165673 14379 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.160127 14314 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.223606 14387 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.190948 14101 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:16.278613 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 209736us
I20260709 12:11:16.279739 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:16.230742 14343 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:16.282663 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:16.274608 14245 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
W20260709 12:11:16.284191 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 75821.9us
W20260709 12:11:16.284739 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 187947us
I20260709 12:11:16.221549 14298 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.295993 14379 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.223244 14423 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.290853 14345 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.226121 13924 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:16.313689 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.320981 10584 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:16.318352 14379 raft_consensus.cc:2764] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:16.325376 14379 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:16.327066 10579 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:16.327827 10587 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:16.328884 14387 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.334208 10438 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:16.301596 14245 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.310531 14426 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.361745 14349 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:16.364930 14414 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9.
I20260709 12:11:16.365712 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.396498 10579 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:16.397675 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.398895 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.354254 14270 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:16.355523 14361 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:11:16.421092 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.409246 10506 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:16.433531 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.435433 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:16.440840 14418 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:11:16.445187 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:16.446581 10513 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:11:16.456813 10478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.666s (SENT) [suppressed 4 similar messages]
I20260709 12:11:16.492249 10510 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:16.497671 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.395542 14426 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:16.504040 14428 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:11:16.507499 14314 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:16.510236 14361 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.408679 14349 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.408814 13640 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.515463 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:16.516139 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:16.516930 14245 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.527426 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.528469 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:16.419822 14416 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:16.487334 14419 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.239s	user 0.001s	sys 0.001s
W20260709 12:11:16.531057 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 251417us
I20260709 12:11:16.562178 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.563529 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.564270 10587 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:16.565275 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:16.566468 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:16.570884 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.571805 10579 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:16.544446 14352 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.008s
I20260709 12:11:16.545444 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:16.531817 14376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.030s	sys 0.042s
I20260709 12:11:16.555599 10582 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:16.559207 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:16.561131 14421 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.576007 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:16.578938 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:16.586649 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:16.555794 14356 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:16.588192 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.594542 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:16.595803 10584 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:16.583000 14376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.031s	sys 0.042s
I20260709 12:11:16.602072 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.579627 13924 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.592428 14421 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.599332 14416 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:16.543715 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46158.4us
I20260709 12:11:16.594839 14431 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:16.613925 10578 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.617401 10582 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:16.592483 14152 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.614722 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.600083 14361 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.651885 10580 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:16.653196 10506 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:11:16.654685 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:16.590298 14352 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.612802 14298 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.661545 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.680191 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.680588 14352 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.680823 14350 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.681190 14352 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:16.681396 14350 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:16.681953 14352 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:16.682639 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.662254 10582 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:16.664112 14421 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:16.666100 14193 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.000s
I20260709 12:11:16.659778 13967 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 12:11:16.668901 14349 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:16.617410 14431 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.698519 14431 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:16.657406 14361 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.660957 14379 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:16.653122 13924 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.648044 14416 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:16.676052 14322 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.007s
W20260709 12:11:16.677430 14359 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.689s	user 0.011s	sys 0.005s
I20260709 12:11:16.677886 14343 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.713784 14343 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:16.685073 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:16.715283 10579 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:16.687112 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:16.705446 13924 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.688651 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:16.713989 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.691845 14368 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:16.719746 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:16.659626 14422 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.277s	user 0.003s	sys 0.000s
I20260709 12:11:16.654434 14376 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:16.691541 14193 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.694736 10585 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:11:16.723062 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2301) took 1080 ms. Trace:
I20260709 12:11:16.672256 14426 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:16.702293 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 245528us
I20260709 12:11:16.689354 14291 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.721735 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.723470 10585 rpcz_store.cc:276] 0709 12:11:15.642694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.644397 (+  1703us) service_pool.cc:224] Handling call
0709 12:11:16.694188 (+1049791us) spinlock_profiling.cc:243] Waited 941 us on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:11:16.694412 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.722762 (+ 28350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.722999 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2163968}
I20260709 12:11:16.730255 10585 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:16.730546 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.732504 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.732936 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:16.734710 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.734949 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.738103 10578 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:16.725378 10507 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:16.688315 14152 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:11:16.743583 10509 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:16.745451 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.723469 14349 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:16.725831 13967 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:16.748564 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.751155 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:16.751765 10579 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:16.742468 14201 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:16.754567 14425 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:16.755807 14152 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:16.756228 14152 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:16.756618 14425 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:16.757027 14425 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:16.757258 14152 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.757862 14425 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.759178 14425 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:16.759817 14393 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:16.753563 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:16.739293 14233 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:16.739151 14352 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:16.764112 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:16.765702 13924 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.767129 14361 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:16.718892 14322 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.758764 14152 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:16.742285 14291 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.721299 14359 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.751101 13967 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.780622 14193 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.745088 14245 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:11:16.781208 14393 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:16.808558 14393 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:16.808993 14393 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:16.748134 14368 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:16.785679 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:16.786162 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.813599 14325 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:16.783229 14416 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:16.801487 13967 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:16.795646 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:16.819377 10579 raft_consensus.cc:606] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.819813 10579 raft_consensus.cc:614] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:16.820048 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.821985 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:16.822489 14202 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:16.802969 11216 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:16.804092 14322 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.804584 14359 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:16.822896 14339 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:16.823756 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:16.823865 14298 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:16.845453 14339 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:16.831499 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c882875c377649acb233035d47b8132e"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:16.846235 14339 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:16.846412 10579 raft_consensus.cc:606] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.846952 10579 raft_consensus.cc:614] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:16.795251 13997 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:16.845674 14298 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:16.863628 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:16.862690 14202 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:16.873260 14298 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:16.800951 14375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.004s	sys 0.011s
I20260709 12:11:16.830637 14291 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:16.882136 14202 consensus_queue.cc:1243] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:11:16.873726 14245 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:11:16.867476 14224 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:16.888636 14245 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:16.888821 14224 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:16.889334 14245 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:16.847461 14350 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:16.877077 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 171068us
W20260709 12:11:16.891233 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.919175 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.922717 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:16.929072 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:16.932260 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:16.879657 14375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.004s	sys 0.011s
I20260709 12:11:16.938100 14439 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:16.890959 14224 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.952605 14431 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.954602 10584 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:16.938155 14182 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:16.927923 14351 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:16.959784 14351 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.916262 14245 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.964818 14245 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:16.966334 14368 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:16.922120 14322 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:16.973299 14393 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:16.978945 11218 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:11:16.983460 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51319.8us
I20260709 12:11:16.982826 14416 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:11:16.983464 14218 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:16.981724 13939 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:16.989806 14242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.284s	user 0.025s	sys 0.064s
I20260709 12:11:16.987514 14322 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:16.998203 14242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.026s	sys 0.064s
I20260709 12:11:16.994390 14218 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:17.001495 14322 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:17.007472 14218 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:16.998013 13939 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.014756 14322 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.017257 14431 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:17.027494 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:17.018009 14393 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:17.023159 14351 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.021804 14218 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:17.045619 14350 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:17.052407 14311 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.065311 14311 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.067777 14311 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.070096 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:17.140442 14431 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:17.141535 14431 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:17.078488 13939 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:17.335423 14351 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.336257 14351 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.337491 14349 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:17.292480 14101 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.117487 13967 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:17.338133 14349 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:17.340404 14379 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.335423 14298 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:17.355266 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:17.344719 14101 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.353536 14351 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.356113 14379 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:17.480486 14325 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.000s
I20260709 12:11:17.416347 14439 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:17.529583 14351 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:17.530484 14351 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:17.562454 10508 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:17.450716 14350 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.604677 14416 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:17.605139 14350 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.605523 14416 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.608384 14416 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.609097 14350 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.563887 14431 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:17.690974 14339 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.839s	user 0.008s	sys 0.009s
I20260709 12:11:17.719074 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:17.714135 14311 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.722697 11221 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:17.714807 14416 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:17.593345 14439 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.721382 14339 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:17.839346 14431 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:17.948464 14349 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:17.949155 14101 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.933578 14350 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:17.937780 14379 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:17.804414 14311 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:17.886873 14416 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.001246 14101 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:18.005548 14447 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.645s	user 0.001s	sys 0.000s
I20260709 12:11:18.005085 14101 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.013722 14351 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.016786 14416 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.016849 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:18.021734 10432 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:18.021066 14322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.020s	sys 0.035s
I20260709 12:11:17.965106 14431 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.000485 14350 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.032598 14349 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.026218 13967 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:18.026245 14322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.021s	sys 0.035s
I20260709 12:11:18.036561 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.038283 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.040656 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.042959 10432 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.033257 14349 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.037299 14416 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.050732 14416 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.051685 14350 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.045929 14439 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.046700 14311 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.041898 14298 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.048210 14431 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.049746 14339 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.056372 14101 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.049871 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.053419 14416 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:18.044560 10432 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1795) took 1241 ms (client timeout 1536 ms). Trace:
I20260709 12:11:18.053907 14349 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:18.068138 10432 rpcz_store.cc:269] 0709 12:11:16.802168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.016347 (+1214179us) service_pool.cc:224] Handling call
0709 12:11:18.043564 (+ 27217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.077446  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:18.096887 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2396) took 1726 ms. Trace:
I20260709 12:11:18.097436 10584 rpcz_store.cc:276] 0709 12:11:16.370135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.602880 (+232745us) service_pool.cc:224] Handling call
0709 12:11:16.651133 (+ 48253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.955217 (+304084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.096555 (+1141338us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400213254. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:18.096795 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697253120}
I20260709 12:11:18.106264 14322 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.107239 14322 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.110514 14322 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.113551 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.115275 14439 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.118826 14416 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.118947 14439 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.119660 14416 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.114689 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.117228 14282 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:18.124217 14282 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:18.107457 11216 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 468) took 1420 ms (client timeout 1589 ms). Trace:
W20260709 12:11:18.128736 11216 rpcz_store.cc:269] 0709 12:11:16.686694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.686841 (+   147us) service_pool.cc:224] Handling call
0709 12:11:18.107257 (+1420416us) spinlock_profiling.cc:243] Waited 85.8 ms on lock 0x7b5400821c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:18.107385 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197309696}
I20260709 12:11:18.113639 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:18.130719 11220 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:18.131606 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.134202 14282 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.136384 14439 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.155766 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.156553 10432 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:18.155426  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:18.158227 10438 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:18.158957 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.159955 10434 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.162020 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:18.162317 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.163414 10434 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:18.164470 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.165336 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.165367 10438 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:18.166038 10432 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:18.154801 10325 maintenance_manager.cc:419] P 92c087afe23c499b9e821ff6f85b7e2a: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033647
I20260709 12:11:18.167302 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.122217 14416 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.168226 10438 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:18.162379 14392 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:11:18.164276 14311 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.163107 14351 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.184203 14311 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.184366 14351 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.161881 14101 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:11:18.186537 14311 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.172638 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:18.187502 10434 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:18.190717 14339 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.191242 14311 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.176242 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.176874 10438 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:18.193125 10431 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:11:18.178843 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.169951 14450 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.174759 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.194512 10433 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:18.201299 10437 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:18.201855 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.202271 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.203024 10437 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.162923 14431 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:18.205545 14431 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.199179 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:18.200933 14450 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.202605 10433 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:18.206732 10430 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:18.199949 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.209226 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.209831 10431 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.209882 10432 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.207115 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.191305 14339 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.210740 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.211156 10437 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:18.216620 11220 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:18.217044 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.217152 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.214932 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:18.218425 10438 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:18.218441 10433 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:11:18.220214 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.221268 14450 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.216048 10431 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:18.228567 10437 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:18.237425 10431 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.248092 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.249047 10438 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 7.
I20260709 12:11:18.211130 14351 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.261336 10430 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:18.263926 14362 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:18.207875 14311 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.273710 14311 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.266021 14450 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:18.273797 10434 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:18.266707 14351 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.250669 14349 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.288333 14349 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.284914 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:18.287770 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.290418 10433 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:18.252082 14350 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:11:18.298604 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.299283 10579 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:11:18.251936 13967 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:11:18.299806 14349 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.286947 14450 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:18.296180 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.300705 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.301066 10584 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.301397 10579 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:18.301578 14450 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.302629 10436 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:18.296335 10583 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:18.302943 14450 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:18.294570 14455 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.297405 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.304219 10581 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:18.303802 14455 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.305965 10506 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:11:18.306293 14455 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.307090 14311 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.307256 14458 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.307685 14311 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.307854 14458 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.307272 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.309808 14351 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.312170 14351 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.293748 14239 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.310103 10506 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:18.313102 14239 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.313911 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.315093 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.315436 14239 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.300467 14349 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.316277 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.316304 14349 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.317664 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.318635 14349 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.310024 14311 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.321174 14458 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:18.324463 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 75714.8us
I20260709 12:11:18.325955 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.327844 10507 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:18.326197 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.328806 10581 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:18.329631 10431 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:18.330744 14339 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.332145 10583 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:18.332974 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.334470 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.335047 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:18.335316 10583 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:18.336616 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.336318 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.337797 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.339172 11212 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:18.340319 10579 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:18.325594 14431 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.346902 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.334812 13967 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.347668 10587 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:18.347958 13967 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.348711 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.349344 10507 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.350206 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.346448 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.351564 10583 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:18.351963 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.352680 10579 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:18.359872 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.361166 10583 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:18.361428 10506 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:18.362567 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.363268 10506 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:18.350234 13967 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.367295 14462 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:18.344072 14349 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.371925 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.364233 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:18.378257 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.379405 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.380457 10512 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:18.368041 14431 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.384985 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.393672 10587 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:18.394323 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.397167 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.397495 14431 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.397922 10509 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:18.398921 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.399788 14454 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.399966 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.400667 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:18.401458 14339 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.401577 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.401976 14339 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.393340 14300 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:18.403937 10508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1880) took 1871 ms (client timeout 1908 ms). Trace:
I20260709 12:11:18.400115 14431 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.402768 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.404431 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.406808 10578 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:18.406903 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.407442 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.400415 14454 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:18.404393 10508 rpcz_store.cc:269] 0709 12:11:16.532434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.558864 (+ 26430us) service_pool.cc:224] Handling call
0709 12:11:18.403737 (+1844873us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b540036ea54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:18.403862 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250969984}
I20260709 12:11:18.414779 10578 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:18.417996 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.415066 10580 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:18.415316 10583 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.415603 10585 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:18.415788 10509 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:18.421171 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:18.417728 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.422304 10508 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:18.423552 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.424547 10508 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:18.425969 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:18.427052 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.427709 10507 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:18.429167 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.429750 10579 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:18.426344 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.416527 14462 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:18.419554 10578 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.438007 10578 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:18.434962 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.439078 10506 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:18.436527 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.447686 14457 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:18.449124 10583 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:18.449451 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:18.446552 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 115228us
I20260709 12:11:18.403551 14300 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.456348 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.457228 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:18.457592 10508 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:18.430917 14431 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:18.462674 10578 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:11:18.457898 10582 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:18.434928 14270 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.451295 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:18.461043 10579 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:18.462575 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.464640 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.465559 10508 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:18.448661 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.467892 10578 pending_rounds.cc:85] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:11:18.409057 14349 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.417958 14213 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:11:18.423899 14239 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:11:18.441119 14417 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:18.452164 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.496119 10583 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:18.475303 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:18.497431 10580 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:18.465346 10509 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:18.478863 10578 pending_rounds.cc:107] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:11:18.423776 14455 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:11:18.482903 10584 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:18.487110 10403 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:18.510442 10403 leader_election.cc:403] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:11:18.487378 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:18.488631 10582 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
W20260709 12:11:18.512494 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 86234us
I20260709 12:11:18.515365 10508 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:18.513511 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:18.468523 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.448172 14457 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.490556 14456 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:18.499159 10578 raft_consensus.cc:2889] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:18.473806 14182 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:18.512924 14461 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:11:18.507296 14213 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:18.489509 14362 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:18.529820 10508 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:11:18.463898 14464 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:11:18.521492 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:18.492748 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:18.536936 10507 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:18.506798 14349 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.492281 14465 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:18.495891 14417 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:18.520092 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51661.3us
I20260709 12:11:18.519639 13967 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.548584 14417 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.490065 14270 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.539492 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:18.540408 14469 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:18.538481 14462 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.526731 14456 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.557106 10504 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:18.558166 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.526772 14457 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:18.528827 14431 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:18.552035 14239 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:18.533145 14182 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.534812 14213 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:18.535012 14362 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:18.568660 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.558434 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.564721 10507 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:18.570770 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.573603 14458 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:18.538813 14465 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.576697 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.578977 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:18.561215 14349 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:18.562683 14450 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:18.588930 11218 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:18.561290 14270 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.574779 14431 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.595121 14431 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:18.596405 14182 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:18.590339 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76914.8us
I20260709 12:11:18.560563 14339 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.561100 14456 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:18.607224 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:18.584810 14465 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:18.608634 14431 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.611496 14431 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.574550 14362 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:18.575920 14469 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:18.615825 14469 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.616776 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:18.588949 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.585745 14239 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:18.585592 14351 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:18.592940 14450 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:18.584154 14458 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:18.577502 14101 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:18.631592 10512 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2276) took 2825 ms (client timeout 2084 ms). Trace:
I20260709 12:11:18.631711 14101 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:18.632177 14101 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:18.632068 10512 rpcz_store.cc:269] 0709 12:11:15.803888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.804284 (+   396us) service_pool.cc:224] Handling call
0709 12:11:18.629539 (+2825255us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b54000daa54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:18.629671 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5922531584}
I20260709 12:11:18.599779 14454 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.635725 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1884) took 2053 ms. Trace:
I20260709 12:11:18.636180 10504 rpcz_store.cc:276] 0709 12:11:16.582036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.591312 (+  9276us) service_pool.cc:224] Handling call
0709 12:11:16.591728 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:18.557628 (+1965900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.635430 (+ 77802us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54001982d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:18.635658 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4519490816}
I20260709 12:11:18.601296 14349 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:18.626717 14463 raft_consensus.cc:2764] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:11:18.583882 14311 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.585768 14392 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:18.640921 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 114326us
W20260709 12:11:18.627004 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.627084 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:18.628026 14239 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:18.639621 14349 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.642169 14311 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.640249 14392 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:18.638684 14458 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:18.630524 14450 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:18.629315 14351 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:18.648233 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:18.657706 10429 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:18.656701 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:18.650261 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.652000 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:18.654100 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 117529us
I20260709 12:11:18.654049 14462 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:18.627593 14213 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.658677 14351 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.700500 14311 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.701025 14350 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:18.701526 14350 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:18.701246 14311 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:18.653744 14431 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:18.672679 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:18.702915 14350 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:18.718629 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46065.7us
I20260709 12:11:18.719447 14350 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:18.719887 14471 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:18.720959 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:18.722998 10505 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:18.731837 13967 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:18.689190 14362 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.732533 14458 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.735984 14362 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:18.739532 14213 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:18.683197 14239 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.202371  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:18.795943 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.753057 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:19.117249 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 364162us
W20260709 12:11:18.789851 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.659209 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:18.860721 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.711400 10504 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:18.968575 14270 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:18.802716 14300 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.656324 14392 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:18.687498 14423 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.795392 14456 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:18.657424 14457 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.266414 10587 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:19.260439 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:18.971709 14431 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:18.751179 14458 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:18.819940 14450 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.271653 10582 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:19.273711 14450 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:18.754606 14463 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:11:19.274250 14431 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.262655 14456 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.275671 10578 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:18.756387 14464 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:18.799355 14465 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.280038 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.280931 10430 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:18.675207 14475 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:19.282605 14457 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:19.283331 14379 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.000s	sys 0.015s
W20260709 12:11:18.989351 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 332141us
I20260709 12:11:19.283670 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.284708 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.285703 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.286736 11216 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:19.104411 11221 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:19.122785 14471 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:19.132722 14300 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:19.288631 14300 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.289199 14300 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:19.290290 14101 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.018s
I20260709 12:11:19.127946 14270 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.290957 14101 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.293339 14270 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:19.137127 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 477778us
I20260709 12:11:19.293790 14101 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.294939 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:19.296902 14128 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:19.296968 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:19.297410 14128 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.146894 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:19.297926 14128 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.167152 10429 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:19.299402 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:19.299516 14475 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:19.300158 10429 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.274909 14463 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.301069 14463 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:19.173486 10432 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:19.282616 14465 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:19.304683 10432 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:19.175076 10431 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:19.175855 14460 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.176592 14462 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.181289 10433 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:19.186702 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:19.187254 10436 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:19.190851 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.213654 11220 raft_consensus.cc:1217] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Deduplicated request from leader. Original: 7.8->[8.9-8.9]   Dedup: 8.9->[]
I20260709 12:11:19.208055 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:19.232534 10333 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 222) took 28383 ms (client timeout 15000 ms). Trace:
I20260709 12:11:19.206226 13967 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:11:19.236709 10438 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:19.240981 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:19.248530 14417 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:19.283828 14379 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.015s
I20260709 12:11:19.285763 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:19.068151 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:19.309055 11220 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:11:19.313302 14462 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.315835 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:19.316579 14460 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.317262 10437 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:19.318385 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:19.318760 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:19.319676 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 252053us
W20260709 12:11:19.320978 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 174539us
I20260709 12:11:19.321594 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:19.321841 14464 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:19.322521 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:19.322880 14464 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:19.323485 14455 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:19.300101 14475 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:19.319778 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 133419us
I20260709 12:11:19.324213 14466 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:19.324959 14270 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:19.363821 14270 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:19.364387 14270 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:19.364886 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:19.325350 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:19.367460 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.368819 13967 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:19.369436 13967 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:19.369635 14239 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:19.325834 14471 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:19.326874 14450 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:19.328645 14456 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:19.329056 14457 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:19.330803 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:19.334602 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:19.337512 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:19.339136 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.340540 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:19.341246 14469 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:19.343815 14473 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:19.348531 14417 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.349635 10585 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:19.351534 10431 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:19.352456 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:19.359786 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.360710 10578 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:19.366743 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:19.316148 14462 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:19.384398 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.386543 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.388633 10511 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:19.389192 14456 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:19.390555 14456 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.392491 10506 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.395100 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.395313 10512 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:19.385005 14455 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:19.396251 10505 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:19.397015 14455 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.399355 14416 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:19.388578 14457 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.399791 14416 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:19.401070 14416 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.401712 14128 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.403133 14379 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.403942 14379 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.406417 14379 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.376482 14450 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.407431 14450 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:19.370774 13967 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.412981 14460 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.415149 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:19.417663 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:19.418512 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62181.8us
I20260709 12:11:19.419844 14339 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:19.420356 14339 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:19.421535 14339 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.357281 14464 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:19.428557 14128 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:19.308415 10333 rpcz_store.cc:269] 0709 12:10:50.849126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:50.868072 (+ 18946us) service_pool.cc:224] Handling call
0709 12:11:19.232127 (+28364055us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:14.426513 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:14.697844 (+271331us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:14.697902 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:11:14.697922 (+    20us) tablet.cc:662] Decoding operations
0709 12:11:14.698898 (+   976us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:14.699003 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:14.699031 (+    28us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:11:14.699903 (+   872us) tablet.cc:701] Row locks acquired
0709 12:11:14.699953 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:11:14.700139 (+   186us) write_op.cc:270] Start()
0709 12:11:14.700321 (+   182us) write_op.cc:276] Timestamp: P: 1783599074700062 usec, L: 0
0709 12:11:14.700341 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:11:14.701100 (+   759us) log.cc:844] Serialized 3266 byte log entry
0709 12:11:15.486579 (+785479us) op_driver.cc:464] REPLICATION: finished
0709 12:11:15.550974 (+ 64395us) write_op.cc:301] APPLY: starting
0709 12:11:15.551126 (+   152us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:15.551368 (+   242us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:15.551516 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:15.558838 (+  7322us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:15.573209 (+ 14371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:11:15.580748 (+  7539us) write_op.cc:312] APPLY: finished
0709 12:11:15.593949 (+ 13201us) log.cc:844] Serialized 65 byte log entry
0709 12:11:15.594184 (+   235us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:15.596562 (+  2378us) write_op.cc:454] Released schema lock
0709 12:11:15.602851 (+  6289us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:15.623404 (+ 20553us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:15.623458 (+    54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3802,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":64038,"apply.run_cpu_time_us":4352,"apply.run_wall_time_us":72348,"num_ops":8,"prepare.queue_time_us":795832,"prepare.run_cpu_time_us":28954,"prepare.run_wall_time_us":733139,"raft.queue_time_us":195147,"raft.run_cpu_time_us":20032,"raft.run_wall_time_us":50961,"replication_time_us":785680,"thread_start_us":1202211,"threads_started":4,"wal-append.queue_time_us":518316,"wal-append.run_cpu_time_us":13194,"wal-append.run_wall_time_us":1433934}]]}
I20260709 12:11:19.433022 14466 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:19.439157 10506 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:19.440145 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117872us
I20260709 12:11:19.425104 14464 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:19.443182 14462 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:19.443692 14462 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.444120 14462 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:19.445933 10505 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:19.402768 14416 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:19.401717 14457 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:19.449862 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 83228.2us
I20260709 12:11:19.456341 10429 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:19.460395 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:19.467967 10511 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
W20260709 12:11:19.474357 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.478677 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:19.475669 14417 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.480734 14362 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:19.481209 14417 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.483349 14349 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:19.484385 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.496546 14417 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.500550 13946 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:19.501052 13946 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:19.503163 14487 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:19.503692 14487 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:19.505980 10479 connection.cc:256] RPC call timeout handler was delayed by -1.7135892709998188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 12:11:19.506497 10479 connection.cc:256] RPC call timeout handler was delayed by -1.6910892709997825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
I20260709 12:11:19.507344 14486 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.508100 14486 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.507961 14463 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.482836 14270 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:19.506803 10479 connection.cc:256] RPC call timeout handler was delayed by -1.6811892709997664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 12:11:19.502239 13946 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:19.512625 10479 connection.cc:256] RPC call timeout handler was delayed by -1.5938892709998527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 12:11:19.513525 10479 connection.cc:256] RPC call timeout handler was delayed by -1.5821892709998338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 12:11:19.513856 10479 connection.cc:256] RPC call timeout handler was delayed by -1.4345892709998225s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 12:11:19.514143 10479 connection.cc:256] RPC call timeout handler was delayed by -1.4030892709997715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 12:11:19.514430 10479 connection.cc:256] RPC call timeout handler was delayed by -1.3265892709998752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20179999999999998s for a response
W20260709 12:11:19.514720 10479 connection.cc:256] RPC call timeout handler was delayed by -1.2995892709998316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
W20260709 12:11:19.515041 10479 connection.cc:256] RPC call timeout handler was delayed by -1.2995892709998316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
W20260709 12:11:19.515331 10479 connection.cc:256] RPC call timeout handler was delayed by -1.2320892709999498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
I20260709 12:11:19.517273 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.511650 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:19.520741 10582 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:19.523408 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.523546 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:19.523954 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.524216 10506 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:19.524432 10433 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:19.524657 10513 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:19.483807 14349 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.525388 14349 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.523564 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.527645 10512 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:19.529549 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73180.2us
I20260709 12:11:19.530392 14417 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:19.531137 14417 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.531289 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:19.536677 14351 consensus_meta.cc:287] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.840s	user 0.009s	sys 0.020s
I20260709 12:11:19.537232 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:19.537585 10436 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:19.533509 14417 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.550460 10585 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:19.541553 14475 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.542099 14463 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.543073 14487 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:19.594918 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 134692us
W20260709 12:11:19.584224 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:19.574162 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 105257us
I20260709 12:11:19.583722 10429 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 6.
I20260709 12:11:19.473618 14239 consensus_queue.cc:1059] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 12:11:19.545176 14350 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.825s	user 0.020s	sys 0.000s
I20260709 12:11:19.580361 14486 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:19.547116 14351 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.599654 10578 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:19.600806 14487 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.601951 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:19.607874 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:19.611630 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.610780 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:19.613489 10512 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:19.619855 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.637467 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:19.637998 14475 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.638512 10583 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:19.642262 10432 raft_consensus.cc:2410] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 12:11:19.603025 14473 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:19.643501 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:19.626180 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:19.644461 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.645594 10433 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:19.577258 10336 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 5 to 6, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 6 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:19.648387 10430 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:19.612512 14270 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:19.653344 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.580021 14392 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.654485 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.656854 11220 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:19.656461 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:19.667212 10585 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:19.669956 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.670047 10435 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:19.670360 10336 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 7 to 8, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 8 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:19.672977 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:19.664270 14491 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:11:19.675482 10336 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 4 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:19.691512 14464 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.694164 14464 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.700595 10434 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:19.679714 14462 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.663522 14350 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:19.717355 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.717505 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.718030 14350 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:19.718540 14213 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.943s	user 0.012s	sys 0.009s
I20260709 12:11:19.667176 14469 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:11:19.720425 14469 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:19.720881 14469 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:19.722000 14469 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:19.726274 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 113111us
I20260709 12:11:19.652231 14473 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.727528 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:19.728781 14463 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:19.729480 14463 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:19.718535 14350 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.735381 14213 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.736150 14213 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.737672 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.737802 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:19.738574 14213 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:19.739471 14491 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:11:19.743386 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.750321 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.753815 10431 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:19.761976 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:19.772374 14490 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:11:19.773355 14469 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:19.777408 14466 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:19.777904 14466 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:19.779646 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001696c0 after lost signal to thread 9264
W20260709 12:11:19.781307 10534 debug-util.cc:398] Leaking SignalData structure 0x7b080024b660 after lost signal to thread 10370
I20260709 12:11:19.782099 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:19.766961 14462 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.775213 14300 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.787926 14351 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.788195 10510 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:19.789441 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.767935 13967 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:19.790561 10511 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:19.789403 14300 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.790809 13967 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.793803 13967 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.796670 14392 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:19.730760 14463 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:19.783064 10437 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:11:19.802202 10585 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
W20260709 12:11:19.824808 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3373909740000727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 12:11:19.825424 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3247909740000523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
I20260709 12:11:19.749029 14473 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:19.835101 10534 debug-util.cc:398] Leaking SignalData structure 0x7b080024b100 after lost signal to thread 10462
W20260709 12:11:19.835549 10534 debug-util.cc:398] Leaking SignalData structure 0x7b080024b500 after lost signal to thread 10463
I20260709 12:11:19.817612 14270 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:19.842249 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 98511.9us
W20260709 12:11:19.847962 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2976637579999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
W20260709 12:11:19.848615 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2869909740002186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
I20260709 12:11:19.829396 14392 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.842717 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:19.850868 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.825984 14466 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.859258 14466 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:19.851100 14462 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:19.866838 10553 connection.cc:256] RPC call timeout handler was delayed by -1.08539097400012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 12:11:19.867331 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0475909740000588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 12:11:19.871953 10584 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:19.874856 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.875905 10512 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:19.865774 10336 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:11:19.880894 10534 debug-util.cc:398] Leaking SignalData structure 0x7b0800244940 after lost signal to thread 10537
W20260709 12:11:19.881273 10534 debug-util.cc:398] Leaking SignalData structure 0x7b0800244a40 after lost signal to thread 10538
I20260709 12:11:19.884371 10510 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:19.887830 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08002b0020 after lost signal to thread 10611
W20260709 12:11:19.888182 10534 debug-util.cc:398] Leaking SignalData structure 0x7b080037fc60 after lost signal to thread 10612
I20260709 12:11:19.864648 14392 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.896351 10336 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 4 to 7, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 7 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:11:19.898346 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08002fdbe0 after lost signal to thread 11248
W20260709 12:11:19.898684 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4120 after lost signal to thread 11249
I20260709 12:11:19.865898 14351 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.906136 10506 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:19.907384 10511 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:19.910015 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.912328 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.913730 13946 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:19.914002 14460 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:19.914499 13967 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:19.916539 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08003dec20 after lost signal to thread 14242
I20260709 12:11:19.893163 14416 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.856199 14450 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.868218 14270 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.914036 14423 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.922263 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:19.925068 10534 debug-util.cc:398] Leaking SignalData structure 0x7b0800283c60 after lost signal to thread 14375
I20260709 12:11:19.886772 14462 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:19.926149 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.926313 14416 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:19.928061 14462 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:19.930200 14270 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:19.933074 14495 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:11:19.935070 14498 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:11:19.935774 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.936646 10507 raft_consensus.cc:2368] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:19.936969 13967 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.938946 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:19.939397 13967 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:19.939842 10587 raft_consensus.cc:2368] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:19.940773 14423 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.947228 10511 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:19.951617 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:19.952160 14497 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:11:19.953888 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 59478.9us
I20260709 12:11:19.965559 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:19.929370 14416 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:19.914680 14460 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.982013 10437 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
W20260709 12:11:19.984684 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58666.8us
I20260709 12:11:19.998189 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:19.995204 11221 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:20.007427 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:20.007645 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 143523us
I20260709 12:11:19.924247 14351 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:19.957969 14349 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.023993 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.026358 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:20.027732 14493 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.116s	user 0.004s	sys 0.001s
I20260709 12:11:20.027824 10586 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:20.028717 10336 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 3 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 5 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:20.023831 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.030560 10511 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:20.032869 14351 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.014022 14423 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.042857 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.043265 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:20.043869 10583 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:20.044273 10578 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:20.044546 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.045369 14450 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:20.045892 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:20.046747 10585 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:20.047564 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.044284 14486 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.049475 10586 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:20.049974 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.050890 14213 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:20.052268 10336 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 5 to 6, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 6 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:20.060031 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:20.062899 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:20.066390 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:20.055121 14213 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.067610 14213 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:20.067127 14351 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:20.071281 14351 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.071717 14351 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:20.064359 14486 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.078094 10585 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:20.079012 14486 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.087046 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:20.087934 10578 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:20.089102 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.089522 14349 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:20.090858 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.094547 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.095175 14487 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:20.097569 14487 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:20.098002 14487 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:20.086885 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.100821 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.103112 10586 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.109082 14392 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.109579 10583 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:11:20.111758 14500 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:20.114468 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.115538 11220 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:20.115936 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.115765 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.065203 14450 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:20.115762 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:20.120514 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 54643.5us
I20260709 12:11:20.150243 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.151403 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.123533 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:20.125980 14339 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.152644 14339 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.153383 10578 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:20.127640 14460 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.128374 14502 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:20.155517 14502 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:20.155911 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.156683 14502 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.157580 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.158048 14502 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.158586 10504 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:20.129020 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:20.131307 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 105067us
I20260709 12:11:20.132901 14487 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.160308 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.160723 14339 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.160444 14487 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:20.164383 14450 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:20.165032 14450 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:20.165439 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.166152 10507 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:20.166065 14450 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.170080 14423 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:20.170593 14423 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.171517 14423 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.170100 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.172737 10504 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:20.174480 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.175704 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.063843 14350 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.177863 14392 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:20.178296 14392 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.178435 14350 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.178731 14392 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:20.146881 11212 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
W20260709 12:11:20.198819 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52795.2us
W20260709 12:11:20.200610 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:20.202591 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.149842 14213 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.204213 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:20.207101 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 146347us
I20260709 12:11:20.211745 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.231966 10435 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:20.231377 10505 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
W20260709 12:11:20.232844 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.233645 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.234338 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.235044 10431 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:20.235988 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.240667 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:20.241720 11212 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:20.236788 10438 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
I20260709 12:11:20.246140 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.255162 10438 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:20.232589 14350 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.257361 10513 raft_consensus.cc:2368] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:20.256958 10432 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:20.257684 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.259884 10435 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:20.261580 10324 maintenance_manager.cc:643] P 92c087afe23c499b9e821ff6f85b7e2a: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 2.041s	user 0.182s	sys 0.064s Metrics: {"bytes_written":29168,"cfile_init":1,"dirs.queue_time_us":64951,"dirs.run_cpu_time_us":523,"dirs.run_wall_time_us":11080,"drs_written":1,"lbm_read_time_us":21589,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":2,"lbm_write_time_us":58698,"lbm_writes_10-100_ms":3,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":63,"thread_start_us":122577,"threads_started":2,"wal-append.queue_time_us":117611}
I20260709 12:11:20.261610 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.264156 10431 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:20.270304 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.270939 10325 maintenance_manager.cc:419] P 92c087afe23c499b9e821ff6f85b7e2a: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8497 bytes on disk
I20260709 12:11:20.272565 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.273885 10324 maintenance_manager.cc:643] P 92c087afe23c499b9e821ff6f85b7e2a: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":397,"lbm_reads_lt_1ms":4}
I20260709 12:11:20.274554 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.277371 10437 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 12:11:20.278538 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.279421 10432 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:20.285487 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.277758 10437 pending_rounds.cc:85] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 5
I20260709 12:11:20.274622 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.286270 10435 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:20.291312 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.286687 10437 pending_rounds.cc:107] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:11:20.292496 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:20.292898 10437 raft_consensus.cc:2889] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:20.295045 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.295743 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:20.300338 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:20.301620 10431 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:20.301504 14213 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:20.293123 14351 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.302150 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.294414 14460 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:20.320053 14460 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.303566 14463 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:20.299074 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 91007.1us
I20260709 12:11:20.308856 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.310441 14518 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:20.299624 14510 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:20.326589 14510 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:20.312636 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 138485us
I20260709 12:11:20.280611 14350 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.298367 14450 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:20.326954 14392 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.341852 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:20.344102 14392 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.345269 10580 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:20.327241 14510 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.327684 14518 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:20.348901 14518 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.352221 14351 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.315210 14423 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.323940 14460 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.352710 14512 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:20.321605 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.329700 14463 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.336501 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.338017 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.338387 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:20.339571 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.332202 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:20.335423 14450 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.361266 14423 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.358561 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.368515 10438 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:20.368424 10437 raft_consensus.cc:2376] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9. Re-sending same reply.
I20260709 12:11:20.397804 14460 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.398577 14460 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.357334 14350 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:20.401067 14460 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:20.402011 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.406759 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.402156 10431 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:20.407960 10438 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:20.402735 14351 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:11:20.405994 10435 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:20.386686 14450 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:20.409794 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116214us
I20260709 12:11:20.405900 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.402199 14423 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:20.415537 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 94040.2us
I20260709 12:11:20.417930 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.417708 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.418684 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:20.421703 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 89636.4us
I20260709 12:11:20.409596 14351 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:20.424494 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.423424 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.429586 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:20.439425 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:20.440712 14392 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:20.441293 14392 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:20.441833 14392 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:20.431957 10438 raft_consensus.cc:2376] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8. Re-sending same reply.
I20260709 12:11:20.447902 10432 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:20.449026 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.450738 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.425969 14522 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:20.454551 14522 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:20.455160 14522 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:20.425993 14351 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:20.425565 14423 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:20.458825 14423 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.475229 14466 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.476087 14466 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.479733 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.480682 10429 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:20.449893 14463 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.489311 10434 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:20.491276 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.491989 14466 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.493955 10403 leader_election.cc:400] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:20.495370 14460 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:20.497917 10403 leader_election.cc:403] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:20.497193 10434 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.500412 14350 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:20.507575 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:20.508570 10429 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:20.501605 14521 raft_consensus.cc:2764] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:11:20.510826 14521 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:20.511052 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71806.4us
I20260709 12:11:20.511472 14521 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:20.512553 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.512063 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.513780 14525 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:20.514212 14525 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.514451 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:20.520411 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.535539 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:20.514567 14525 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.540459 14522 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.543295 14463 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:20.543819 14463 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.544246 14463 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:20.545188 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:20.512048 14521 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:20.545884 14466 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:20.547991 14466 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:20.549068 14239 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:20.549477 14239 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:20.550447 14239 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.553642 14515 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:20.554085 14515 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:20.554742 14515 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.518764 14510 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.557525 11212 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.559860 14350 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:20.561003 14510 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.561172 14350 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:11:20.572183 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 123286us
I20260709 12:11:20.561605 14350 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:20.582311 14239 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:20.603116 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.601058 14213 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:11:20.609530 14522 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.611384 14213 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:20.614625 14450 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.616101 14392 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.617899 14450 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.618307 14392 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.607191 14351 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.619426 14350 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.620987 14351 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:20.621141 14350 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.622437 14460 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:20.623450 14524 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:20.618453 14521 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.625900 14466 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:20.626456 14466 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:20.629025 10584 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:20.627077 14518 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:11:20.684893 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 81698.6us
I20260709 12:11:20.686652 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:20.691354 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 109684us
I20260709 12:11:20.680228 14518 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:20.699707 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 154659us
I20260709 12:11:20.688122 14416 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:20.704109 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.707505 14501 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:20.708819 14501 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:20.709197 14501 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:20.705935 14463 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.713361 14463 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.710263 14501 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.716472 14501 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:20.717342 14510 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.717294 14416 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:20.723426 14487 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.724083 14487 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.719237 14416 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:20.692358 14466 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.728626 14466 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:20.731244 14534 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.731870 14534 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.733883 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.733978 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.734613 10507 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:20.734727 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.734870 10504 raft_consensus.cc:1813] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Handling vote request from an unknown peer 33d77e3063684586a737bccf358786d8
I20260709 12:11:20.735352 10505 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.735713 10504 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:20.735986 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.736711 10511 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:20.736830 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.738034 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.718273 14510 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:20.740477 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106488us
W20260709 12:11:20.744870 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.701679 14518 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:20.753664 10505 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:20.726866 14487 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.760154 14537 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:20.760401 14515 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.763195 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.764722 14534 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.764931 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.765419 14534 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:20.763229 14213 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:20.766252 14213 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:20.768076 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.768442 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.768764 10582 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:20.769105 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.767333 14213 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:20.770264 10513 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:20.771152 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.771523 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.772240 10587 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:20.772776 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.773437 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:20.774391 10580 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:20.775652 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.776041 10579 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:11:20.757664 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.783591 10507 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:20.770283 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.780057 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.744956 14463 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.794806 10587 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.796238 10513 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:11:20.797645 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:20.798882 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.787659 14351 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:20.797528 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 110984us
I20260709 12:11:20.788381 14515 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:20.790678 14450 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.802114 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:20.802878 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 95745us
I20260709 12:11:20.795441 14423 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:20.795347 14533 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:20.810799 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.801632 14392 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:20.788148 14525 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.800809 14463 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.811631 13946 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:11:20.804409 14534 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:11:20.807003 14524 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:20.837451 10552 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.838598 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:20.824486 14213 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.803565 14351 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.809088 14423 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.811607 14533 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.849666 13946 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:20.838346 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:20.856328 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:20.856391 14416 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.856738 14486 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:20.816864 14239 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:11:20.859635 14531 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:20.833379 14392 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:20.860816 14531 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:20.861521 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.861191 14392 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.832058 14539 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:20.853940 14423 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:20.864233 14416 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.853906 13946 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:20.865332 14350 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.865902 13946 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.867511 13946 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.867616 14510 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.868530 14351 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.854529 14533 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:20.869899 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61662.2us
I20260709 12:11:20.870767 10507 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:20.871155 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:20.813813 14349 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:11:20.861207 14531 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:20.814927 14450 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:20.840080 14524 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:20.829175 14521 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:20.889626 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.888427 14351 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:20.893021 14532 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:20.893772 14450 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.875015 14525 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:20.895423 14532 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:20.896018 14532 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:20.897661 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.856469 14515 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:20.904412 14463 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.883625 14531 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:20.909510 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:20.939710 10507 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:20.913167 10513 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:20.883622 14524 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:20.894603 14450 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:11:20.943022 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:20.938088 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67112.9us
I20260709 12:11:20.915108 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:20.865926 14350 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.941730 10552 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:20.875448 14486 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:20.913080 14351 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:20.941332 14521 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:20.959816 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.958169 14416 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:20.960467 14416 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.960878 14416 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:20.946293 14349 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:20.966363 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 128500us
I20260709 12:11:20.946108 14541 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:20.965271 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106770us
I20260709 12:11:20.968453 10552 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:20.972131 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.975493 14351 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.978547 14349 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:20.980931 14349 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:20.996353 10580 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:20.965648 14521 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.003499 14531 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.005494 14530 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.039196 14530 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.042179 14530 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.003243 14546 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:21.044596 14546 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.045141 14546 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:21.013736 14339 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:11:21.047716 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:21.048590 14339 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:11:21.053424 14548 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:11:21.015514 13946 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:11:21.055007 13946 raft_consensus.cc:3055] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:21.055521 13946 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:20.964154 14532 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.061028 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.061863 10513 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:21.067879 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:21.019865 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:21.029739 14416 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:11:21.068934 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 52701.7us
I20260709 12:11:21.070312 14416 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.071092 14416 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:21.071609 14416 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.072014 14416 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:21.072172 14541 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:21.075006 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:21.076305 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:21.070994 14547 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:21.077122 14547 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:21.077929 14213 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:21.078429 14350 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:21.078740 14213 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:21.087102 14213 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:21.087544 14213 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:21.089514 14423 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.092875 14416 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.080765 14521 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.081804 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:21.109428 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:21.111660 10507 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:21.114106 14530 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:21.114578 14530 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.115020 14530 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.115113 14533 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.116866 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:21.117342 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:21.050774 14339 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:21.118775 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:21.013006 14349 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.118153 11212 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:21.085290 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:21.086153 14239 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:21.078233 14547 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.105746 14450 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.106400 14486 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:11:21.108494 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:21.107872 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.056856 13946 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.119532 11218 raft_consensus.cc:1813] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Handling vote request from an unknown peer 33d77e3063684586a737bccf358786d8
I20260709 12:11:21.118952 14339 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:21.147307 11218 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:21.123649 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.148662 10511 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:21.124429 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.149636 10510 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:21.129282 11220 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:21.134817 10513 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.135444 10506 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:21.140749 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:21.125674 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.135157 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:21.155313 10432 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:21.143388 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:21.131299 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:21.144526 11221 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:21.145057 11215 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:11:21.222384 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.225240 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:21.226286 10434 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.187734 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:21.125185 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.145995 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:21.229175 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:21.136102 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:21.236776 10438 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10. Re-sending same reply.
I20260709 12:11:21.237035 11212 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:21.229848 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:21.238786 10429 raft_consensus.cc:2376] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11. Re-sending same reply.
W20260709 12:11:21.241012 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 53460.4us
I20260709 12:11:21.238116 10437 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:21.162950 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:21.233058 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 422294us
W20260709 12:11:21.142148 10553 connection.cc:256] RPC call timeout handler was delayed by -1.4391569260001233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 12:11:21.230921 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:21.129978 14537 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:21.263186 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:21.264334 10435 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10. Re-sending same reply.
W20260709 12:11:21.270104 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:21.270550 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3842569260000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 12:11:21.271895 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3743569260000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 12:11:21.272459 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2465569260000393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:11:21.272878 10553 connection.cc:256] RPC call timeout handler was delayed by -1.1898569259999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
I20260709 12:11:21.271157 10431 raft_consensus.cc:2376] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8. Re-sending same reply.
W20260709 12:11:21.273387 10553 connection.cc:256] RPC call timeout handler was delayed by -1.120556926000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:11:21.274977 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0935569260000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
W20260709 12:11:21.275761 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0827569260000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 12:11:21.263167 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:21.139201 14423 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.140885 13946 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:21.284693 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:21.143226 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:21.286798 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:21.121869 14239 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.283371 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.285866 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:21.289392 10403 leader_election.cc:400] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:21.291635 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:21.294395 10403 leader_election.cc:403] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:11:21.294467 10435 raft_consensus.cc:2376] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8. Re-sending same reply.
I20260709 12:11:21.293726 10510 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:21.129515 14450 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.299823 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:21.303648 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.297868 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:21.120667 14521 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.168s
I20260709 12:11:21.265311 14537 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.261813 14392 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.278429 14525 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:21.301304 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:21.300647 10430 raft_consensus.cc:2376] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10. Re-sending same reply.
I20260709 12:11:21.120067 14349 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:21.310735 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 233579us
I20260709 12:11:21.292640 14350 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:21.306890 11178 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.065s (SENT)
I20260709 12:11:21.314512 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.315833 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.317194 10579 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:21.319182 11212 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:21.321893 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.295089 14239 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:21.316388 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:21.324903 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.325800 10579 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:21.327211 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:21.327697 14518 consensus_meta.cc:287] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.548s	user 0.014s	sys 0.000s
I20260709 12:11:21.328182 14518 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.329313 10584 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:21.330315 10433 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:11:21.132306 14486 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.185s
I20260709 12:11:21.334820 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.335476 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.336082 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:21.341369 10583 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.333153 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:21.310221 14525 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.311918 14423 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:21.309878 14537 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.339205 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.344686 14349 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:21.355855 10434 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:21.340413 10578 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:21.366634 10435 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:21.367305 14533 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:21.385983 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45318.3us
I20260709 12:11:21.332818 14350 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:21.371927 14525 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:21.371660 14515 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:21.372221 14423 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:21.374095 14450 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:21.397356 11178 leader_election.cc:336] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.654s (SENT)
I20260709 12:11:21.362639 14530 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.413223 10431 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:21.403086 11216 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:21.415383 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.416379 10580 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:21.330507 14392 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.424044 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.448462 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:21.421866 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:21.420151 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.451176 10587 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:21.370929 14546 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.461537 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.472419 10585 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:21.468891 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:21.477548 10587 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:21.477065 14534 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.446102 14510 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:11:21.515328 14539 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.521584 14530 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.522514 14423 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.523109 14350 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.473901 14518 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:21.527143 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 209339us
I20260709 12:11:21.532991 11213 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:21.531028 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 246410us
I20260709 12:11:21.535512 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:21.567281 10513 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:21.533844 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:21.572702 10553 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:21.574448 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:21.527019 14349 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.585088 10552 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:21.590775 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:21.591084 10578 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:21.592896 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.596535 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.517385 14213 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.518134 14539 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:21.601783 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.608349 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:21.583595 14515 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:21.609320 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.612002 10504 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:21.599336 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 180605us
I20260709 12:11:21.533279 14534 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.621912 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:21.617545 10553 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.536s (SENT)
W20260709 12:11:21.615338 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 195094us
I20260709 12:11:21.622741 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:21.622375 10513 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:11:21.624604 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:21.599999 14533 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:21.574003 14553 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:11:21.587512 14450 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:11:21.616715 14524 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.666s	user 0.015s	sys 0.000s
I20260709 12:11:21.658471 10509 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:21.660213 10403 leader_election.cc:400] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8.
I20260709 12:11:21.661125 10585 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:21.633764 10584 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:21.632835 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.634719 10578 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:21.598371 14558 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:21.665838 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49058.6us
I20260709 12:11:21.688541 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:21.690094 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 81851.9us
I20260709 12:11:21.621201 14556 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:21.597959 14521 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:11:21.596130 14349 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.626121 14518 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:21.695928 10585 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:21.698522 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:21.697170 10432 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.001s
I20260709 12:11:21.701340 10432 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:21.682168 14558 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.708045 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:21.685632 14559 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.712832 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:21.699585 14416 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.747071 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.676398 14557 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:11:21.685359 14524 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.702726 14460 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9.
I20260709 12:11:21.706938 14522 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:21.709045 14558 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:21.766862 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 82813.5us
I20260709 12:11:21.724530 14546 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:21.775664 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 158144us
I20260709 12:11:21.781728 14553 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:21.782191 14556 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:11:21.768446 14567 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:11:21.789036 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:21.725582 14463 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:11:21.773641 10510 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:11:21.750809 14566 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:21.768249 14576 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:21.802851 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 115659us
I20260709 12:11:21.768630 13946 raft_consensus.cc:2764] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:11:21.778390 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.580s (SENT) [suppressed 27 similar messages]
W20260709 12:11:21.842324 10583 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.500s	user 0.022s	sys 0.000s
W20260709 12:11:21.810159 14541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.010s	sys 0.021s
I20260709 12:11:21.797340 14556 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:21.849552 10583 raft_consensus.cc:2410] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:21.848980 10585 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:21.833717 14339 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.752113 14559 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:21.865804 14550 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.800s	user 0.000s	sys 0.007s
I20260709 12:11:21.865612 14213 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:21.866408 14556 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:21.846823 14541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.012s	sys 0.021s
I20260709 12:11:21.832326 14566 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:21.871508 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:21.890950 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.834257 14392 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:21.879272 10587 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:21.892530 11177 leader_election.cc:336] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.654s (SENT)
W20260709 12:11:21.880316 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 188985us
I20260709 12:11:21.893169 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:21.834213 14562 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:11:21.871440 14518 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:21.808745 14553 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:21.902813 14569 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:21.903506 14349 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:21.903756 14549 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:21.904307 14545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.000s	sys 0.018s
I20260709 12:11:21.905093 14568 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:11:21.905390 14577 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:21.906085 14555 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:11:21.907073 14582 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:21.907478 14239 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.568s	user 0.013s	sys 0.000s
I20260709 12:11:21.911746 14562 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:21.912786 14560 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
I20260709 12:11:21.883671 14539 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:21.899313 14213 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:11:21.892164 14566 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.915530 14556 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:21.923480 14572 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:11:21.925639 14524 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.926630 14585 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.065649 14585 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:21.931979 14559 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:21.915272 14534 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:21.934947 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1947) took 1031 ms. Trace:
I20260709 12:11:22.032824 14239 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.069681 10505 rpcz_store.cc:276] 0709 12:11:20.903988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:20.904128 (+   140us) service_pool.cc:224] Handling call
0709 12:11:20.904913 (+   785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:20.956241 (+ 51328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:20.956294 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:20.969115 (+ 12821us) log.cc:844] Serialized 26 byte log entry
0709 12:11:21.011432 (+ 42317us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:21.013018 (+  1586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:21.013134 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:21.697665 (+684531us) spinlock_profiling.cc:243] Waited 81.1 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:21.934578 (+236913us) raft_consensus.cc:1685] finished
0709 12:11:21.934610 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:21.934857 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41108,"raft.queue_time_us":47544,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":159,"spinlock_wait_cycles":186454912,"thread_start_us":27505,"threads_started":1,"wal-append.queue_time_us":81899}
I20260709 12:11:21.869014 14576 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.072870 14239 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:22.075419 10506 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:21.939956 10513 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:21.940296 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.940928 10504 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:21.941097 14525 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
I20260709 12:11:21.941663 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.943387 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:21.944836 14533 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:21.945699 14537 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.573s	user 0.014s	sys 0.001s
W20260709 12:11:21.955644 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 184661us
W20260709 12:11:22.004631 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:22.082370 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.005465 14557 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:22.047971 14515 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:22.083401 14557 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.084208 14515 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:22.085398 11177 leader_election.cc:336] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.830s (SENT)
I20260709 12:11:22.085891 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:22.006270 14571 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:21.949453 14547 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.005s	sys 0.008s
W20260709 12:11:22.096169 14547 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.005s	sys 0.008s
I20260709 12:11:22.096153 14571 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:22.022553 14545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.000s	sys 0.019s
W20260709 12:11:22.030056 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 163342us
I20260709 12:11:22.086320 14537 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.098071 14545 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:22.099069 14545 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:22.099529 14537 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:22.099941 14588 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:22.100409 14588 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:22.031193 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 136370us
I20260709 12:11:22.100219 14537 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:22.101403 14588 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.094734 14525 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.102236 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:22.103492 10581 raft_consensus.cc:2368] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:22.106765 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:22.033080 14562 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:22.028620 14569 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.041446 14539 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.003733 11217 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:11:22.087793 10512 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:22.089051 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:22.038599 14423 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.007s	sys 0.007s
I20260709 12:11:22.095168 14572 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:22.100885 14534 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:22.074586 14576 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:22.112125 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:22.152813 10578 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:22.112464 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:22.113138 10586 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:22.113871 10587 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:22.098719 14571 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:22.115149 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:22.131529 14423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.008s
I20260709 12:11:22.158615 10580 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:22.120541 14582 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:11:22.125061 14350 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.012s	sys 0.046s
W20260709 12:11:22.166110 14350 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.013s	sys 0.046s
W20260709 12:11:22.167954 14586 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:11:22.102860 14588 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:22.171558 14534 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:22.172415 14534 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:22.046366 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 268002us
I20260709 12:11:22.175594 14534 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.146701 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:22.123801 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.183398 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:22.184111 10504 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:22.185799 10586 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:22.132292 14572 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:22.142335 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53414.4us
I20260709 12:11:22.165544 14594 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:11:22.188932 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:22.190969 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 65004us
I20260709 12:11:22.194028 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:22.191159 10580 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
W20260709 12:11:22.191601 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.102259 14545 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:11:22.200294 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:22.200337 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:22.210947 10511 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:22.205478 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:22.208442 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:22.133531 14539 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.163012 14549 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.234789 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:22.221724 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:22.239532 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:22.249171 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:22.334913 10432 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:22.329833 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:22.253888 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:22.118100 14518 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:11:22.352607 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 130918us
I20260709 12:11:22.344530 10507 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:22.186154 14582 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:22.190429 14585 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:22.468937 10432 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:22.498914 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 290719us
I20260709 12:11:22.500265 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:22.200305 14350 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:22.498972 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:22.525532 10430 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:22.132645 14569 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.467156 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:22.209959 14501 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:22.467605 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:22.246806 14463 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:22.524250 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:22.247815 14534 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:22.549259 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:22.555686 10507 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.249823 14510 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:22.350589 14557 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:22.525353 14423 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:22.576496 10511 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:22.585587 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:22.549816 14578 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:22.587426 14349 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:22.567766 14510 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:22.618677 10506 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:22.562057 14537 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:11:22.620563 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:22.461404 14549 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.641491 14577 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.598129 14553 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:22.571973 14350 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.594697 14423 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:22.622112 14510 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:22.331990 14562 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.574898 14572 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:22.690251 11221 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:22.683235 10507 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:11:22.700743 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 461255us
I20260709 12:11:22.701776 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:22.700769 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:22.708057 11215 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:22.716786 10553 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.589s (SENT)
I20260709 12:11:22.683979 14531 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:22.722056 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:22.733038 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:22.683507 14598 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:22.687810 14553 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:22.717267 14595 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.551s	user 0.001s	sys 0.000s
I20260709 12:11:22.719149 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:22.747219 14599 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:22.755784 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:22.763727 10582 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:22.716463 14575 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:11:22.654304 14582 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.770287 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:22.769392 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:22.763121 14573 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:11:22.739885 14531 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:22.777136 14602 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:11:22.777626 14613 raft_consensus.cc:2764] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:11:22.778301 14606 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:22.768189 14547 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:22.785229 10430 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:22.791324 14423 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.686432 14577 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:22.796694 10506 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:22.766113 14450 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:22.773687 14558 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.988s	user 0.011s	sys 0.005s
I20260709 12:11:22.769398 14539 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:22.809950 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 93221.1us
W20260709 12:11:22.808259 10578 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.002s
I20260709 12:11:22.814360 10578 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:22.820495 10506 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:11:22.820523 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50347.5us
I20260709 12:11:22.822595 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:22.763211 14553 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:22.827968 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.808380 14547 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:22.859242 10553 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.632s (SENT)
I20260709 12:11:22.812418 14537 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.866483 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:22.818284 14590 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:22.830243 14599 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:11:22.819445 14558 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:22.878355 10504 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.000s
I20260709 12:11:22.880501 14339 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:22.880714 14596 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:11:22.882690 14531 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:22.885218 14608 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:11:22.874071 14613 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:22.860657 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:22.923239 10504 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:22.913172 14450 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:11:22.932624 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75751.6us
W20260709 12:11:22.936561 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.009s	sys 0.009s
W20260709 12:11:22.939786 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.009s	sys 0.009s
W20260709 12:11:22.940250 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 728102us
I20260709 12:11:22.889488 14537 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:22.938723 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 78143.9us
I20260709 12:11:22.952337 14423 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.952529 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:22.953073 14423 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:22.954651 14618 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:11:22.956139 14423 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:22.923761 14549 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:22.963634 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:22.964346 10432 raft_consensus.cc:1813] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Handling vote request from an unknown peer 33d77e3063684586a737bccf358786d8
I20260709 12:11:22.964870 10432 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:22.935766 14569 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.041553 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:22.918830 14590 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.039251 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:22.942123 14599 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:23.040575 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.045033 10429 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:22.948145 14619 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.046427 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.054140 10437 raft_consensus.cc:2368] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12.
I20260709 12:11:23.047281 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.055307 10436 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:23.056291 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.057514 10438 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:11:23.057951 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.058578 10432 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:23.064743 14603 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:11:23.067216 14628 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.067914 14577 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.067827 14628 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.070363 14628 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.073141 10432 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:23.074783 14590 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.040068 14549 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:23.053395 14616 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.261s	user 0.000s	sys 0.002s
I20260709 12:11:23.066795 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2341) took 1128 ms. Trace:
I20260709 12:11:23.048655 14599 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:23.082523 14599 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:23.041306 14547 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.091156 14549 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.097146 14558 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.099579 14549 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.104604 14486 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:23.105118 14486 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:23.106178 14486 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.107623 14486 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:23.108038 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:23.108623 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:23.103423 14534 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.002s
I20260709 12:11:23.110132 10433 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:11:23.112902 14599 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.061371 14619 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.115351 14350 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.115569 14599 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.116182 14619 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.116396 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2436) took 1733 ms. Trace:
I20260709 12:11:22.961347 14530 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
W20260709 12:11:23.113456 14556 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.051s	user 0.009s	sys 0.005s
I20260709 12:11:23.066502 14510 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.128361 10429 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:23.116844 10435 rpcz_store.cc:276] 0709 12:11:21.382792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.387953 (+  5161us) service_pool.cc:224] Handling call
0709 12:11:21.388399 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:21.388912 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:21.618118 (+229206us) log.cc:844] Serialized 14 byte log entry
0709 12:11:22.047174 (+429056us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:23.109827 (+1062653us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b2c00000f48. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:23.110280 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:11:23.110480 (+   200us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:23.110519 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.110591 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:23.116094 (+  5503us) raft_consensus.cc:1685] finished
0709 12:11:23.116198 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.116320 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":264482,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":23082,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":5143,"spinlock_wait_cycles":35816192,"thread_start_us":504634,"threads_started":2,"wal-append.queue_time_us":925105}
I20260709 12:11:23.080336 10509 rpcz_store.cc:276] 0709 12:11:21.938640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.944390 (+  5750us) service_pool.cc:224] Handling call
0709 12:11:21.944761 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:22.096317 (+151556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.096352 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:22.096939 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:11:22.211780 (+114841us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:22.499254 (+287474us) log.cc:844] Serialized 14 byte log entry
0709 12:11:22.813355 (+314101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.047380 (+234025us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540038f4d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:23.047436 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:23.066414 (+ 18978us) raft_consensus.cc:1685] finished
0709 12:11:23.066533 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.066698 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13545,"prepare.queue_time_us":319024,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":14965,"raft.queue_time_us":139126,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":13698,"spinlock_wait_cycles":329970304,"thread_start_us":428080,"threads_started":2,"wal-append.queue_time_us":453338}
I20260709 12:11:23.130896 10508 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:23.135941 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.136704 10433 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:23.140183 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.141621 10429 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:23.147419 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.147825 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.148408 10438 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:23.151319 14580 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:23.151803 14580 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:23.152493 14510 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:23.152782 14580 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.153544 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.153967 14580 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:23.154409 10437 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.157039 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.157356 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.158422 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:23.158764 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.159482 10504 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:23.160753 10512 raft_consensus.cc:2410] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:11:23.150781 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.157796 10509 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:23.166033 14466 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:23.166538 14466 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:23.167481 14466 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.168625 14466 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:23.169806 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.172185 14518 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:23.172811 14518 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:23.173453 14591 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.649s	user 0.000s	sys 0.010s
I20260709 12:11:23.175434 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.176853 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2347) took 1234 ms. Trace:
I20260709 12:11:23.177268 10508 rpcz_store.cc:276] 0709 12:11:21.942878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.943060 (+   182us) service_pool.cc:224] Handling call
0709 12:11:23.176642 (+1233582us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400360454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:23.176754 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2400126848}
I20260709 12:11:23.177650 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.178020 10512 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:23.178385 10504 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:23.179927 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.179932 10438 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:23.182281 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:23.182888 10509 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:23.184485 10506 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:23.141407 14553 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.179493 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:23.160290 14588 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.989s	user 0.015s	sys 0.000s
I20260709 12:11:23.200973 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.201906 10438 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:23.203693 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45503.5us
I20260709 12:11:23.205118 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:23.191573 14518 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.208681 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.216053 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:23.228688 10429 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:23.224009 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.177277 14578 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:23.216900 14556 raft_consensus.cc:2764] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:11:23.231031 10436 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.234907 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.206609 10433 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:23.199631 14559 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:23.200441 14510 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:23.245294 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.250425 14562 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:23.209810 14627 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:23.191915 14423 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:11:23.262405 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:21.355126  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:23.265901 11218 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:23.231623 14553 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:23.255916 14566 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.173s	user 0.010s	sys 0.006s
I20260709 12:11:23.292476 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2473) took 1085 ms. Trace:
I20260709 12:11:23.277974 10513 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:23.270828 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:23.295835 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:23.296005 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.293061 10582 rpcz_store.cc:276] 0709 12:11:22.183516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.183686 (+   170us) service_pool.cc:224] Handling call
0709 12:11:22.237685 (+ 53999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:22.764395 (+526710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.268724 (+504329us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54001d60d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:23.268958 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208718592}
I20260709 12:11:23.297866 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:23.298925 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 107712us
I20260709 12:11:23.262952 14350 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:23.302862 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.263106 14599 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:23.302856 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.239446 14578 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.301167 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:23.314580 14549 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:23.315156 14549 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:23.315555 14549 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:23.319526 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48797.8us
I20260709 12:11:23.320621 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:23.321625 14597 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:23.327476 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.328682 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.329533 10429 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:23.265363 14510 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.299469 14627 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:23.312271 14460 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.310182 14569 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.311297 14558 raft_consensus.cc:2764] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:11:23.349663 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.300643 14566 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.358531 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:23.309093 14350 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.341049 14559 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:23.360723 14623 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:23.363992 14634 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:11:23.363664 14623 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:23.367228 14623 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
W20260709 12:11:23.367659 14633 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:11:23.313205 14599 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:23.356396 14632 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.147s	user 0.001s	sys 0.001s
I20260709 12:11:23.344657 14534 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.341218 14553 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:11:23.382767 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 77850.3us
I20260709 12:11:23.383428 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.384537 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.386155 10429 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:23.387874 10552 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.386080 10436 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:23.351436 14636 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:11:23.404120 10429 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:23.354355 14460 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.401139 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.359287 14569 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:23.411598 10581 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2353) took 2092 ms (client timeout 1647 ms). Trace:
W20260709 12:11:23.412066 10581 rpcz_store.cc:269] 0709 12:11:21.315431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.315554 (+   123us) service_pool.cc:224] Handling call
0709 12:11:23.407496 (+2091942us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54002765d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:23.407738 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807283584}
I20260709 12:11:23.416188 10433 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:23.416872 10436 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 12:11:23.417241 10436 pending_rounds.cc:85] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 4
I20260709 12:11:23.417536 10436 pending_rounds.cc:107] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:11:23.417768 10436 raft_consensus.cc:2889] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:23.406486 14423 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:23.420548 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.437496 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.438691 10438 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:23.435577 10509 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
W20260709 12:11:23.440769 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96035.8us
I20260709 12:11:23.385540 14534 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.443209 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2337) took 1605 ms. Trace:
I20260709 12:11:23.443430 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.446504 10429 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:23.423916 14635 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:11:23.450533 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.442404 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.455531 10433 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.425241 14619 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.428550 14537 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.459190 10429 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:23.463515 10438 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:23.464576 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.465438 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.424391 14350 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.471392 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.446242 10510 rpcz_store.cc:276] 0709 12:11:21.837882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.838135 (+   253us) service_pool.cc:224] Handling call
0709 12:11:21.841324 (+  3189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:21.841567 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:22.048571 (+207004us) log.cc:844] Serialized 14 byte log entry
0709 12:11:22.167500 (+118929us) log.cc:844] Serialized 14 byte log entry
0709 12:11:23.093587 (+926087us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b18002ad420. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:23.176952 (+ 83365us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b18002ad420. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:23.369293 (+192341us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b18002ad420. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c92ad7060 00007f2c92ae9716 00007f2c92afa287 00007f2c92afa208
0709 12:11:23.442831 (+ 73538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:23.442877 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:23.442916 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.442973 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.443144 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32106,"prepare.queue_time_us":617547,"prepare.run_cpu_time_us":191,"prepare.run_wall_time_us":22477,"raft.queue_time_us":27450,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":225,"spinlock_wait_cycles":129327360,"thread_start_us":328885,"threads_started":3,"wal-append.queue_time_us":169139}
W20260709 12:11:23.471786 11217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57892 (request call id 495) took 2333 ms (client timeout 1540 ms). Trace:
W20260709 12:11:23.473075 14637 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:11:23.441615 14558 raft_consensus.cc:2764] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:11:23.477651 10436 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:23.420043 14641 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:23.479429 10430 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:23.480528 10432 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:23.480904 11217 rpcz_store.cc:269] 0709 12:11:21.138683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.140361 (+  1678us) service_pool.cc:224] Handling call
0709 12:11:23.471516 (+2331155us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b540047a9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:23.471673 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280524544}
I20260709 12:11:23.480517 10552 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.443773 14590 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:23.452168 14423 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:23.483527 14515 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:23.507479 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.443945 14579 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:11:23.511516 10433 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.508225 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:23.499603 14339 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.557s	user 0.003s	sys 0.011s
I20260709 12:11:23.521245 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.522521 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:23.523931 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49862us
W20260709 12:11:23.527566 11178 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.164s (SENT)
I20260709 12:11:23.528007 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.528975 10432 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:23.480731 14553 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:11:23.532485 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:23.534200 14638 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:11:23.535070 14566 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.536476 14460 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.536630 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:23.538484 14510 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.538709 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.514051 14641 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:23.539458 14599 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.539523 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.539913 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:23.441414 14627 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.544078 14450 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:23.579525 14525 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:23.580622 14641 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.584318 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.546188 14549 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.549178 10431 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:23.547747 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.553238 10432 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:23.590085 10432 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:23.554093 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.555821 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.481904 14619 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.495857 14598 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:11:23.559825 11178 leader_election.cc:336] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.217s (SENT)
I20260709 12:11:23.594058 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:23.515982 14639 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:23.564350 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.598595 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.599634 10431 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:23.600910 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.569911 10581 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:23.480441 14559 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.571599 11217 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:23.511404 14590 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.608373 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:23.576280 14553 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:23.609869 14515 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:23.611753 14599 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:23.611361 14515 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.612385 14599 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:23.613024 14599 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:23.577953 14588 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:23.615271 14588 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:23.616695 14588 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.620934 14450 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:23.586481 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.591347 10430 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:23.585273 10435 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 2.
W20260709 12:11:23.627856 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43810.2us
I20260709 12:11:23.618747 14588 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:23.630781 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:23.631820 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 121467us
W20260709 12:11:23.637583 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77830.5us
I20260709 12:11:23.640950 10435 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:23.634032 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.653390 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.570890 14578 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.659649 10431 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.660343 10436 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:23.662024 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.662845 10436 raft_consensus.cc:2376] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7. Re-sending same reply.
I20260709 12:11:23.670215 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:23.671988 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:23.673050 11215 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:23.674798 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.597630 14639 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:23.685719 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.681960 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.683310 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:23.687628 10431 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:11:23.687582 11217 raft_consensus.cc:2376] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5. Re-sending same reply.
I20260709 12:11:23.689036 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.690528 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.691226 10431 raft_consensus.cc:2376] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9. Re-sending same reply.
I20260709 12:11:23.692492 10435 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 5: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.693142 11216 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:11:23.665477 10432 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.637442 14598 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:23.652104 14566 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:23.699728 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.705986 10433 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:23.706959 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.708683 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.705039 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.709992 10436 raft_consensus.cc:2368] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:23.700515 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:23.703089 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.231s (SENT)
I20260709 12:11:23.698135 10579 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:23.703393 10432 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:23.713039 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:23.703851 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.714172 10435 raft_consensus.cc:2376] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9. Re-sending same reply.
I20260709 12:11:23.714236 10431 raft_consensus.cc:2376] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11. Re-sending same reply.
I20260709 12:11:23.635630 14619 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:23.715451 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.715945 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.719758 10433 raft_consensus.cc:2376] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8. Re-sending same reply.
I20260709 12:11:23.719117 10435 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:23.723937 10430 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:23.692610 14639 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:23.728103 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:23.724984 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.729290 10431 raft_consensus.cc:2376] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12. Re-sending same reply.
I20260709 12:11:23.730337 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.726619 10435 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:23.726181 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.731062 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.704668 14576 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:23.732352 14450 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.743475 14569 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:23.744961 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:23.745354 14645 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:11:23.745775 11217 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:23.747502 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:23.748040 14609 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:23.748535 14609 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:23.748986 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.749845 14569 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:23.749608 14609 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.750298 14569 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:23.750706 14569 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:23.631170 14423 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.758723 14549 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:23.759366 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56362.3us
W20260709 12:11:23.760769 10513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 1960) took 1569 ms (client timeout 1835 ms). Trace:
I20260709 12:11:23.730897 10431 raft_consensus.cc:2376] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12. Re-sending same reply.
I20260709 12:11:23.701512 14598 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:23.764683 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.771376 10432 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
W20260709 12:11:23.761360 10513 rpcz_store.cc:269] 0709 12:11:22.191450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.211416 (+ 19966us) service_pool.cc:224] Handling call
0709 12:11:23.760557 (+1549141us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b540036f454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:23.760687 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1979937664}
I20260709 12:11:23.771870 10432 pending_rounds.cc:85] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 6
I20260709 12:11:23.772080 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.772388 10432 pending_rounds.cc:107] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:11:23.773348 10432 raft_consensus.cc:2889] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:23.775303 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.776909 10432 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:23.776250 10435 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.768016 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.782413 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.783895 10433 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:23.785113 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:23.786480 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.798098 10431 raft_consensus.cc:2368] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:23.805765 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.733755 14566 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:23.750828 14609 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:23.904305 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.771606 14585 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.843326 14213 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:23.890621 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.882143 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:23.843350 14580 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.004s
I20260709 12:11:23.913013 10512 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:23.913182 10431 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:23.909701 14585 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:23.867734 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.918671 10504 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
W20260709 12:11:23.873863 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 145803us
I20260709 12:11:23.850867 14579 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.882143 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.924616 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.862176 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:23.843828 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.925455 10508 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:23.880092 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.911808 14213 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:23.813779 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.926857 10430 raft_consensus.cc:2368] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:23.843636 14598 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:23.872120 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.928303 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.916167 10436 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:23.882788 10510 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:23.917354 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.871789 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:23.924223 10434 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:23.929780 10433 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:23.930178 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.930439 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.932559 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.932863 11215 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:23.851485 14651 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:23.873297 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.871346 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:23.934965 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:23.937506 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:23.938905 10585 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:23.938858 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.940587 10580 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:23.943302 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:23.944169 11217 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:23.945287 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.946321 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.947935 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.948354 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.963482 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:23.963984 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.964450 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:23.965188 10509 raft_consensus.cc:2393] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:23.967391 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:23.971558 10433 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:23.969563 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:23.976197 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.980792 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:23.981577 10511 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:23.983691 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:23.984298 14649 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:11:23.985698 11218 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:23.990724 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:23.991529 10511 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:23.993134 10585 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:23.993217 10584 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:23.943388 14580 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.996577 10585 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:23.998664 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.004534 14534 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.007230 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:24.028409 11215 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:24.010571 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.013636 14578 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:24.027132 14213 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.036015 14213 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.036634 14534 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.037279 14534 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:24.041523 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.042531 10510 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:23.943605 14651 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.043408 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:24.044263 11217 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:24.045747 14651 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:24.047087 14656 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:11:24.047748 14460 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.043444 10510 raft_consensus.cc:606] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:24.048832 10510 raft_consensus.cc:614] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:23.989920 14510 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:23.972415 13946 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:23.965756 14566 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:23.974535 14598 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:23.938140 14579 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.067646 11213 raft_consensus.cc:2393] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:24.069173 11218 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:24.024649 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.081842 10511 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
W20260709 12:11:24.092721 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.093746 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.095759 10584 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:24.061560 14510 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.100821 10578 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:24.071667 14565 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.061811 14578 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:24.080739 14566 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.033368 14580 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.080096 14650 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:24.115301 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.117108 11215 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:24.109223 14510 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.118162 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:24.120904 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.117802 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.096314 14651 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:11:24.080513 13946 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.125284 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:24.125572 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:24.128391 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 129841us
I20260709 12:11:24.119035 14566 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.087229 14639 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.140251 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.141134 10584 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:24.144389 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:24.128981 14549 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:24.116266 14423 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:11:24.149694 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.150271 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.137495 14651 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:11:24.118801 14565 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.162541 10582 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:24.166949 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:24.168485 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47410.6us
I20260709 12:11:24.168646 14619 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:24.169286 13946 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:24.169340 14619 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.172155 14619 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.176798 14565 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.121189 14650 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.177825 10584 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:24.179044 14650 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:24.179416 10584 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:24.164963 14648 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:24.179927 14648 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:24.154032 14549 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.181535 14549 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.183648 14510 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.184743 14569 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.187284 10432 raft_consensus.cc:2368] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12.
I20260709 12:11:24.187448 14510 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.188580 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:24.195046 14569 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.195390 14585 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:24.207518 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.212113 14580 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.211874 14549 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.213802 14639 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:24.215299 14579 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.144418 14534 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.218247 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:24.218927 14574 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:24.219323 14623 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:24.219791 14574 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.222963 14549 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.232751 14667 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:24.232967 14666 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:24.233817 14667 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:24.162169 14665 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:24.235556 14665 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:24.235942 14665 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:24.235821 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66516.5us
I20260709 12:11:24.237102 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.211560 14578 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.250645 14651 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:24.251442 14651 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:24.255311 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.256500 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:24.257247 11215 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.261250 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.262651 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.260418 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.258751 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.274787 10584 raft_consensus.cc:2393] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:24.278079 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:24.279009 10511 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
W20260709 12:11:24.252802 14588 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.004s
I20260709 12:11:24.282498 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.285195 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:24.286594 10582 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:24.286842 10504 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:24.288089 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:24.288681 11216 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:11:24.289129 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.279982 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:24.252091 14651 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.295689 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.291496 10509 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:24.293804 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:24.294039 11215 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:24.252066 14486 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:11:24.296586 10584 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:24.302436 14486 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:24.302875 14486 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:24.303169 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.304051 10511 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.298220 10512 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:24.304703 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:24.306514 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:24.306272 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:24.309334 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 179534us
I20260709 12:11:24.305400 10508 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:24.310142 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.290570 14213 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:24.311667 14673 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:24.311775 14565 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:24.315758 14565 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:24.316198 14565 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.312455 14673 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:24.311484 14213 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.317054 14671 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:24.317595 14671 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:24.285512 14582 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:11:24.317970 14673 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.319108 14213 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.252468 14667 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.318010 14671 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:24.313325 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.345846 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1958) took 2126 ms. Trace:
I20260709 12:11:24.287515 14556 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.346670 10505 rpcz_store.cc:276] 0709 12:11:22.187242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.187416 (+   174us) service_pool.cc:224] Handling call
0709 12:11:22.235711 (+ 48295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:22.468467 (+232756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.468499 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:24.170685 (+1702186us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b2c0001c5e8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:24.171248 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:11:24.301276 (+130028us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:24.301335 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.301426 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:24.304295 (+  2869us) raft_consensus.cc:1685] finished
0709 12:11:24.304319 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.313562 (+  9243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":259917,"raft.queue_time_us":347370,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":9335,"spinlock_wait_cycles":69960576,"thread_start_us":129509,"threads_started":1,"wal-append.queue_time_us":129000}
I20260709 12:11:24.323323 11177 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:24.333873 10511 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:24.291450 14585 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:24.307332 14510 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:24.367548 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.367892 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.306447 14571 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:24.380000 11216 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
W20260709 12:11:24.369361 14636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.020s	sys 0.016s
W20260709 12:11:24.388249 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 127076us
I20260709 12:11:24.390182 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.366673 14672 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.370260 14510 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:24.387450 14636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.021s	sys 0.016s
I20260709 12:11:24.373086 14562 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:24.367129 14574 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.387077 14549 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:24.383881 14534 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.321259 14648 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.413725 10583 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:24.413652 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:24.425133 10582 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.392668 14580 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 12:11:24.415544 14599 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.007s
I20260709 12:11:24.321233 14486 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.409603 14510 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:24.429965 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 62523.3us
I20260709 12:11:24.431784 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.432196 14460 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.434924 10583 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:11:24.436815 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.414486 14562 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.411865 14672 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.438486 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:24.442058 14562 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.444962 14650 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:11:24.445417 14650 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:24.445799 14650 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:11:24.447516 14669 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:11:24.448256 14585 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:24.448696 14585 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.449071 14585 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:24.449656 14547 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.435655 14599 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.452329 10435 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12.
I20260709 12:11:24.453557 14599 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.456857 14562 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.461477 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.463265 14572 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.466796 14590 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.464916 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:24.439368 14569 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.471458 10583 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.471666 14569 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.474205 14569 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:24.459137 14609 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.536s	user 0.003s	sys 0.011s
I20260709 12:11:24.465545 14562 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.466161 14672 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:24.526895 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.501454 10585 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:24.527300 10582 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:24.487464 10583 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:24.479009 10578 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:24.459524 14648 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:24.534181 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:24.538919 11177 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:24.505065 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.458974 14580 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:24.537901 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76527us
I20260709 12:11:24.548368 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:24.537261 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:24.481590 14659 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
I20260709 12:11:24.554996 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.491222 14460 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:24.500650 14541 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.565744 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:24.556516 14667 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:24.560971 14580 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:24.575923 14580 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:24.576143 10513 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:24.576424 14580 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.558815 13946 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:24.478372 14566 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.475062 14585 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.517975 14651 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:24.579591 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:24.586210 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 196126us
I20260709 12:11:24.508244 14671 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.578890 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:24.561213 14578 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:24.514032 14665 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:24.585114 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46291.7us
I20260709 12:11:24.587603 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.578653 14580 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.592226 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:24.595772 14660 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.507s	user 0.003s	sys 0.000s
I20260709 12:11:24.577160 14673 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:11:24.568555 14541 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.599715 14565 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.602259 14565 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.603422 14510 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.606642 14510 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.608053 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.595652 14671 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:24.612735 10507 raft_consensus.cc:2376] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11. Re-sending same reply.
I20260709 12:11:24.599728 14683 raft_consensus.cc:2764] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:11:24.631621 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:24.617703 14578 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:24.633883 14578 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.635452 10511 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:24.636018 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.636626 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:24.637836 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.639369 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:24.640122 10504 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:24.641050 10507 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:24.619289 14665 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:24.621632 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2489) took 1081 ms. Trace:
I20260709 12:11:24.585996 13946 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:24.643903 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.644979 13946 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.643222 10586 rpcz_store.cc:276] 0709 12:11:23.540430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.543385 (+  2955us) service_pool.cc:224] Handling call
0709 12:11:24.152819 (+609434us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:11:24.153095 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:24.183323 (+ 30228us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:24.310925 (+127602us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.425508 (+114583us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:24.425588 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:24.425662 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.621371 (+195709us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b54001a1154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:24.621432 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.621568 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120378,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":27480,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":4022,"spinlock_wait_cycles":85491328,"thread_start_us":229068,"threads_started":2,"wal-append.queue_time_us":120681}
I20260709 12:11:24.652298 10552 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:24.653048 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65533.5us
I20260709 12:11:24.650915 14585 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:24.624964 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46278.1us
I20260709 12:11:24.625299 14639 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:24.625651 14562 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.625735 14651 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.626305 10579 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:24.626899 14667 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:24.636662 10512 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.657447 14671 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:24.667078 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.669401 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.672722 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:24.674739 10507 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:11:24.675434 10585 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:24.682014 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.683665 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:24.668676 14667 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:24.688323 14667 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.689774 14667 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:24.693387 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:24.693989 10586 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:24.657461 14665 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:11:24.667289 14639 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:24.699999 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.692664 14673 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.716305 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:24.692533 14460 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:24.695384 14672 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:24.692852 14648 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:11:24.703830 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:24.722080 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:24.705082 10403 leader_election.cc:400] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:24.722963 10403 leader_election.cc:403] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:24.718422 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:24.698374 14684 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:11:24.739646 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.740618 10429 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:24.741058 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:24.708799 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.744264 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:24.723870 10434 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:24.724514 14681 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.731382 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:24.757066 10438 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:24.731714 10331 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:24.715889 14639 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:24.736840 10336 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 235) took 31852 ms (client timeout 15000 ms). Trace:
I20260709 12:11:24.736569 14650 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:24.702181 14683 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:11:24.720191 14651 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:24.768949 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.721127 14672 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:24.748477 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43584.7us
I20260709 12:11:24.750110 10435 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:11:24.753839 10434 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:24.755177 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:24.697041 10585 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:24.762276 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.778239 10580 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:24.762661 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.779415 10584 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:24.719636 10512 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:24.752766 14534 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:24.748652 10429 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:24.770586 10582 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:24.771353 10430 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:24.720896 14460 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:24.779197 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.716537 14671 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:24.783267 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.786108 10578 raft_consensus.cc:1240] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 2. Current term is 3. Ops: []
I20260709 12:11:24.787616 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.788836 10438 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:24.790287 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.788535 10552 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:24.784848 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:24.776746 14672 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.720568 14580 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.829309 14580 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.832520 14541 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.837281 14549 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:24.837752 14549 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:24.796380 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.807399 14599 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:24.766218 10336 rpcz_store.cc:269] 0709 12:10:52.884106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:52.884458 (+   352us) service_pool.cc:224] Handling call
0709 12:10:53.985450 (+1100992us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a49090 00007f2c92ae7f3a 00007f2c9889de9c 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc 00007f2c96e5c392 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:24.736729 (+30751279us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:20.393882 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:20.394566 (+   684us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:20.394630 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:11:20.394648 (+    18us) tablet.cc:662] Decoding operations
0709 12:11:20.395660 (+  1012us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:20.395777 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:20.395814 (+    37us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:11:20.396670 (+   856us) tablet.cc:701] Row locks acquired
0709 12:11:20.396733 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:11:20.396926 (+   193us) write_op.cc:270] Start()
0709 12:11:20.397085 (+   159us) write_op.cc:276] Timestamp: P: 1783599080396853 usec, L: 0
0709 12:11:20.397126 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:11:20.487366 (+ 90240us) log.cc:844] Serialized 3205 byte log entry
0709 12:11:20.772797 (+285431us) op_driver.cc:464] REPLICATION: finished
0709 12:11:20.843496 (+ 70699us) write_op.cc:301] APPLY: starting
0709 12:11:20.843647 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:20.988011 (+144364us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:20.988182 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:21.123565 (+135383us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:21.123909 (+   344us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:11:21.123957 (+    48us) write_op.cc:312] APPLY: finished
0709 12:11:21.124387 (+   430us) log.cc:844] Serialized 81 byte log entry
0709 12:11:21.125874 (+  1487us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:21.126029 (+   155us) write_op.cc:454] Released schema lock
0709 12:11:21.126103 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:21.127040 (+   937us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:21.127074 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3425,"reported_tablets":51,"spinlock_wait_cycles":301534080,"child_traces":[["op",{"apply.queue_time_us":65072,"apply.run_cpu_time_us":19720,"apply.run_wall_time_us":283171,"cfile_cache_hit":30,"cfile_cache_hit_bytes":31365,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":1,"lbm_read_time_us":24910,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":7,"num_ops":8,"prepare.queue_time_us":186084,"prepare.run_cpu_time_us":4974,"prepare.run_wall_time_us":93909,"replication_time_us":375483,"thread_start_us":111682,"threads_started":2}]]}
I20260709 12:11:24.785252 14460 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.807965 14671 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.845216 14673 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:24.845716 14673 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:24.784936 14450 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:24.849017 14650 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:24.809491 14559 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:24.850040 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 681) took 1004 ms. Trace:
I20260709 12:11:24.850302 14559 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:24.850484 10579 rpcz_store.cc:276] 0709 12:11:23.845540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.863842 (+ 18302us) service_pool.cc:224] Handling call
0709 12:11:24.498871 (+635029us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:11:24.499181 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:24.668022 (+168841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.849785 (+181763us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400276ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:24.849970 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295254656}
I20260709 12:11:24.798943 14547 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:11:24.851379 14559 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.766860 14681 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.808911 10331 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 11, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:24.793787 14569 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.791628 14666 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:24.814139 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 422) took 1068 ms. Trace:
I20260709 12:11:24.794274 14683 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:11:24.780894 14651 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:24.855896 10432 rpcz_store.cc:276] 0709 12:11:23.745592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.778009 (+ 32417us) service_pool.cc:224] Handling call
0709 12:11:24.813897 (+1035888us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400435b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:24.814061 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593669376}
I20260709 12:11:24.857172 14651 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:24.820327 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.827154 14525 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:24.859937 14525 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:24.860687 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:24.861052 14525 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.867344 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:24.867667 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:24.868175 11215 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13.
W20260709 12:11:24.878674 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.898386 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:24.882534 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47240.4us
I20260709 12:11:24.802261 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:24.768028 14689 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:24.897111 10404 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:24.901954 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 111749us
I20260709 12:11:24.924535 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.929214 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:24.929905 10513 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:24.931496 11216 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:24.797444 13946 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:24.928331 10586 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
W20260709 12:11:24.937422 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 135267us
I20260709 12:11:24.948382 10437 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:24.866597 14683 consensus_queue.cc:1243] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:11:24.956315 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:24.957085 10584 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:24.958595 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.909044 14515 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.965137 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.908241 14578 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:24.960878 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:24.961990 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:24.962882 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:24.966352 10509 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:24.846588 14673 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.936823 14585 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:24.838591 14549 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:24.981583 10479 leader_election.cc:400] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13.
I20260709 12:11:24.991410 10479 leader_election.cc:403] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:24.949493 14609 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:24.949275 14569 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:24.940717 14541 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.949887 14677 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:24.983574 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:24.948541 14640 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:24.983534 10582 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:24.985970 14605 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:25.015808 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:25.000016 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 203725us
I20260709 12:11:24.987927 14585 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:24.966223 14213 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.977553 14578 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:24.855002 10331 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:25.026166 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.036762 10437 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.028939 10553 leader_election.cc:400] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:25.002954 14646 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:25.038763 14646 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.039204 14646 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.979432 14515 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:24.992233 14681 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:24.976030 14691 consensus_queue.cc:1059] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:11:24.993831 14609 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:25.058053 10580 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:25.034837 14578 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:25.063879 14696 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:11:25.064054 14681 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:11:25.005137 14599 consensus_queue.cc:1175] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 12 status { last_received { term: 12 index: 7 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:11:25.067198 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45455.9us
I20260709 12:11:25.068025 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.016386 14541 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:25.074215 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 138192us
I20260709 12:11:25.025470 14605 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.015628 14673 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.040123 14213 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.044052 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.086920 10586 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:11:25.087378 10432 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.015192 14640 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:25.033634 14585 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:25.009505 14619 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:11:25.080240 10437 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:25.054468 14609 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:25.014747 14689 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.081028 14650 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:25.060024 14582 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:11:25.091943 14640 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.143424 10438 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:11:25.147965 14213 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:25.109143 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.108843 14652 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:11:25.119663 14650 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:25.081089 14239 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:11:25.149360 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.094808 14691 raft_consensus.cc:3055] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:25.093654 14566 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:25.173616 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:25.175349 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 92543.7us
W20260709 12:11:25.177070 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 236114us
I20260709 12:11:25.180262 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:25.030174 10509 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:25.177146 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.196959 10579 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:25.198318 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.199219 10438 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:25.192915 14392 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:25.199925 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.241737 14665 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:11:25.245500 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45659us
I20260709 12:11:25.247190 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.189922 14571 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13.
W20260709 12:11:25.253731 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 275185us
I20260709 12:11:25.170884 14691 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:25.197369 14580 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:25.203428 14681 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:25.120450 14689 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:25.292347 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 308841us
W20260709 12:11:25.294090 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46997.4us
I20260709 12:11:25.296350 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.240084 13946 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:25.310601 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.293697 14574 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:25.319190 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84726.6us
W20260709 12:11:25.317548 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.318646 11178 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:25.302446 14665 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:25.321429 10582 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:25.304719 14667 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:25.325451 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:25.304482 14392 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:25.324267 10582 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:25.324321 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 304473us
I20260709 12:11:25.323391 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.326150 10509 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:25.310487 14569 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:25.307058 14515 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:25.327248 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.333861 10510 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.315248 14530 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:25.330861 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:25.339277 10504 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:11:25.339175 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.341655 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.342355 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.344550 14239 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.344636 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:25.345289 14239 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.344439 14569 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:25.346494 14703 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:11:25.313225 14641 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:11:25.348253 14713 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:25.348855 14713 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.351306 13946 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.327250 14691 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.357306 14691 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:25.359341 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.360006 10508 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.359678 14392 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.342341 10437 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:25.311360 10331 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:25.336352 14530 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:25.336711 14699 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:11:25.380896 14605 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.342434 14510 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:25.318540 14704 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:11:25.310714 14705 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:11:25.366933 14720 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:11:25.340412 14572 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:11:25.438380 14609 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.441449 14713 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.442255 14609 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.384845 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:25.391098 14566 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:25.332001 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.443998 14423 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:25.388509 14558 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:25.444244 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 112651us
I20260709 12:11:25.444440 14423 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:25.326099 14650 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.444692 14558 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.392966 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:25.394212 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:25.451148 10582 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 6: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:25.445291 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.394953 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:25.395241 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:25.405501 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:25.409631 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 86361.7us
I20260709 12:11:25.387789 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.414196 10508 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:25.423935 10331 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:25.426185 14239 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:25.429637 14702 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
W20260709 12:11:25.363919 14700 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:11:25.447392 14486 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.433662 14510 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.441923 14585 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.443331 10509 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:25.449112 14423 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:25.454707 10504 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:25.390352 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.486752 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.488217 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.455298 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.489636 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:25.458966 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.459358 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.490890 10434 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:25.491856 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.492419 10434 raft_consensus.cc:2376] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8. Re-sending same reply.
I20260709 12:11:25.493252 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.493287 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:25.461772 10579 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:11:25.463371 14569 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:25.463271 14652 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:25.402866 14530 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.460863 14566 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:25.464237 14715 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:25.461079 14579 raft_consensus.cc:2764] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:11:25.465449 14667 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:25.463281 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.448002 14689 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:25.475930 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.483605 10510 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:25.458381 10582 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
W20260709 12:11:25.410691 14639 consensus_meta.cc:287] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.003s
I20260709 12:11:25.494582 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:25.525245 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:25.501536 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2644694699999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 12:11:25.540498 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46010.5us
W20260709 12:11:25.480242 14460 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.003s
I20260709 12:11:25.539024 10437 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 5.
I20260709 12:11:25.496850 10429 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.533867 11216 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:25.533936 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 210645us
W20260709 12:11:25.495982 14672 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.630s	user 0.018s	sys 0.000s
I20260709 12:11:25.476903 14510 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.539247 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.525101 14239 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.541023 10434 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.528894 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.546605 10432 raft_consensus.cc:2376] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11. Re-sending same reply.
I20260709 12:11:25.564357 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.565078 10437 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:25.565006 10582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:25.566164 10582 raft_consensus.cc:606] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.537637 14609 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:25.479843 14641 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:11:25.534399 14720 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:25.531935 14605 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:25.497393 14565 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:25.558969 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.535758 14667 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:25.574162 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.570187 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.577419 10432 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:25.577543 10434 raft_consensus.cc:2376] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11. Re-sending same reply.
I20260709 12:11:25.577654 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.578810 10429 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.573163 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.531570 14569 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.533162 14566 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.539294 14572 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:11:25.563809 14599 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:25.535799 14578 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.584160 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.584968 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.589538 14667 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:25.590344 14714 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:11:25.590797 14566 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:25.594551 14569 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:25.580174 14565 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:25.647784 14585 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:25.647954 14565 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:25.647212 10331 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 10, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:25.535550 14715 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.649209 14565 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.649365 10331 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:25.596570 10432 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:25.651382 14715 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:25.652101 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.652842 10432 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:25.653882 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.568121 14588 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.636422 14705 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:11:25.592877 14710 raft_consensus.cc:2764] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:25.567656 14558 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:25.574620 14671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.004s	sys 0.039s
I20260709 12:11:25.548151 14460 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.672062 14667 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.673868 14460 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:25.546567 14639 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.675176 10583 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:25.675767 14562 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:25.676189 14562 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:25.676345 14639 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:25.592680 14706 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.129s
I20260709 12:11:25.677199 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 698) took 1049 ms. Trace:
I20260709 12:11:25.677271 10582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:25.677963 10582 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.607132 10437 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 6.
I20260709 12:11:25.547762 14672 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.560250 14239 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.636422 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:25.628679 14525 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:25.555527 14640 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.660970 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.683247 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.584729 14609 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
I20260709 12:11:25.683969 10432 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:25.670118 14671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.004s	sys 0.039s
I20260709 12:11:25.683998 10437 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.661191 10434 raft_consensus.cc:2376] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10. Re-sending same reply.
I20260709 12:11:25.668901 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:25.588032 14720 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:25.598752 14666 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:25.677733 10583 rpcz_store.cc:276] 0709 12:11:24.627417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.627602 (+   185us) service_pool.cc:224] Handling call
0709 12:11:24.628649 (+  1047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:25.675564 (+1046915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:25.676952 (+  1388us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400335254. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:25.677140 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406228736}
I20260709 12:11:25.686672 11215 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:11:25.691138 10433 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:25.692282 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:25.697572 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.698695 10432 raft_consensus.cc:2410] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:25.696427 10404 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:25.701263 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116531us
I20260709 12:11:25.702244 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:25.691159 14525 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:25.690887 14666 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:25.709489 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:25.702926 14646 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.004s
W20260709 12:11:25.707396 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 142253us
I20260709 12:11:25.702145 14460 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:11:25.714704 10437 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:25.697569 14691 consensus_queue.cc:1059] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:11:25.716177 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:25.718022 10583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
W20260709 12:11:25.719098 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 176533us
I20260709 12:11:25.720518 10583 raft_consensus.cc:606] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.721032 10583 raft_consensus.cc:614] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:25.715559 14673 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.004s
I20260709 12:11:25.723165 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:25.724920 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:25.727018 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:25.730924 11220 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:11:25.732723 11218 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:25.733270 14557 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:25.733698 14557 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:25.731017 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:25.738060 14639 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:25.739004 14639 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:25.737255 14650 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:25.740052 14650 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:25.740367 14239 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.741335 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:25.714815 10331 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 10, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:25.742393 14578 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.739370 14639 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:25.743266 14525 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.743607 14639 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.744642 14339 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:25.745225 14339 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:25.746570 14599 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.747074 14599 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.747499 14588 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:25.723721 14423 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.750648 14423 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:25.714166 14460 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:25.719471 14572 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:11:25.751237 14460 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:25.751407 14572 consensus_queue.cc:1243] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:25.751726 14549 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.714269 14646 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.752102 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:25.753855 14691 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:25.754400 14691 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:25.734603 14557 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.755184 14557 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.755345 14691 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.743041 10331 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 7, term changed from 7 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:25.740427 14650 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:25.760187 10431 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:11:25.753988 14646 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.764137 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.745054 14639 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:25.765723 10512 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:25.768934 11215 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 3.
I20260709 12:11:25.772070 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:25.773375 10553 leader_election.cc:336] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.641s (SENT)
I20260709 12:11:25.773931 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.756749 14691 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.776827 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.746248 14339 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.779006 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.790563 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:25.738453 14562 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.783717 14652 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:25.793006 14650 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.793692 14562 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.793839 14730 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:25.794281 14710 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.794469 14650 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.795112 14710 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.798409 14710 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.802377 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:25.783998 14648 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:25.762060 14666 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.805640 14666 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.762403 14582 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.807397 14582 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:25.810501 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83576.5us
W20260709 12:11:25.835578 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56725.4us
I20260709 12:11:25.837225 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.765272 14605 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.884528 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:25.886660 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49532us
I20260709 12:11:25.889158 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:25.891042 14672 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.892077 10331 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 8, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:11:25.885794 14726 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:11:25.794579 14730 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.835511 14673 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:25.899782 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.905081 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:25.899879 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.902765 10508 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:25.909155 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.910650 10504 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:25.913381 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.916463 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.917630 10510 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:25.848796 14652 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:25.926321 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:25.927796 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.929358 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:25.930193 10509 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
W20260709 12:11:25.894958 14696 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:11:25.894554 14460 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.897337 14730 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:25.849097 14648 raft_consensus.cc:697] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:25.939625 10513 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:25.937160 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.810012 14723 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:11:25.932487 14609 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.952056 14704 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:25.848953 14646 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.091s
W20260709 12:11:25.954356 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45594.3us
I20260709 12:11:25.893919 10331 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 7, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:25.945640 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.956494 10504 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:25.932857 14598 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.962431 14559 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:25.948431 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:25.949255 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:25.925683 14515 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:25.962905 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.924950 14667 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:25.964171 14730 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:25.964658 14731 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:25.965215 14730 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:25.943122 14665 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:25.904048 14605 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.761868 14720 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.944217 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:25.970451 10509 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12.
I20260709 12:11:25.965867 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:25.920966 14673 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:25.951633 14671 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:25.925482 14486 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:11:25.968526 14731 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:11:25.978876 14723 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:25.979971 14673 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:25.969715 14598 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.954694 14640 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:25.997502 11177 leader_election.cc:400] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12.
I20260709 12:11:26.016361 11177 leader_election.cc:403] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:26.021173 14605 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.021878 14541 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.025581 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:25.972779 14710 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:26.031855 14721 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.969002 14665 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:25.998363 14599 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:25.997092 10504 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:26.047936 11221 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:25.967417 14510 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:26.051512 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 85773.3us
I20260709 12:11:26.056524 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:26.060528 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.061569 10437 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:26.018965 14598 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:26.064360 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 136891us
I20260709 12:11:26.066371 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:26.043004 14665 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.068081 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.071398 10434 raft_consensus.cc:2368] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:26.071482 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:26.051342 14667 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.080017 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2503) took 1125 ms. Trace:
I20260709 12:11:26.074158 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.076979 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:26.076999 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.080464 10586 rpcz_store.cc:276] 0709 12:11:24.954386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.964445 (+ 10059us) service_pool.cc:224] Handling call
0709 12:11:24.974814 (+ 10369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:25.087363 (+112549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.079760 (+992397us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540010d1d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:26.079952 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256914688}
I20260709 12:11:26.075769 14605 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:11:26.082614 14640 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:26.085343 11219 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:11:26.066102 14730 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.092463 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.093741 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:26.096513 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:26.069253 14673 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:26.103243 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 141236us
I20260709 12:11:26.103853 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.076584 14598 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:26.105065 10437 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:26.077767 14672 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:26.142253 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.097525 14640 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:26.097539 14731 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:11:26.159950 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48568.5us
I20260709 12:11:26.165170 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.100543 14580 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:11:26.106630 14530 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.003s
I20260709 12:11:26.073222 14710 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:11:26.193178 14530 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.197765 14530 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:26.198519 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.182551 14673 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.204430 14239 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:26.204322 14673 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.204977 14239 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:26.207288 14580 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:26.208963 14580 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.122084 14672 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.211314 14672 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:26.102917 10331 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 5 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:26.181181 10437 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:26.186399 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.215080 14640 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:26.215355 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.215577 14640 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.216099 14640 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:26.075930 14648 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.218760 14339 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.219532 14339 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:26.221570 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.187253 14599 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:26.188915 11213 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 589) took 1331 ms (client timeout 1555 ms). Trace:
I20260709 12:11:26.190044 14582 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:26.198909 14510 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.234035 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.206104 14239 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.184003 14746 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:26.229694 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:26.240064 14650 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:26.092720 14667 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.241319 14650 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:11:26.241716 14740 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:11:26.241879 14650 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:26.242727 14558 consensus_queue.cc:1175] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 13 status { last_received { term: 13 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 13 index: 6 } }
I20260709 12:11:26.243605 14667 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.244128 10586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
W20260709 12:11:26.232174 11213 rpcz_store.cc:269] 0709 12:11:24.857319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.857948 (+   629us) service_pool.cc:224] Handling call
0709 12:11:24.860542 (+  2594us) spinlock_profiling.cc:243] Waited 990 us on lock 0x7b5800023198. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:11:26.188615 (+1328073us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540047a9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:26.188791 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750205184}
I20260709 12:11:26.244814 10586 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:26.245226 10586 raft_consensus.cc:614] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:26.245414 14746 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:26.246958 14650 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:26.238480 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 126713us
I20260709 12:11:26.239274 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:26.239661 11219 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:26.234776 10513 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:26.251370 14650 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.252507 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:26.233937 14541 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.254644 14541 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:26.256280 14741 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:11:26.265662 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:26.267042 10583 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:26.258281 14541 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.183048 14681 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.276295 11219 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:26.276620 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.277845 10437 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:26.267987 14667 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:26.289052 14486 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:26.294102 14673 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.295199 10578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:26.296754 10511 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:26.297324 10511 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:26.268811 14466 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:26.321010 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.325912 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.327003 10513 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:26.329664 10437 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:26.330816 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.331557 10437 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:26.339210 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.342234 10437 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:26.348419 14569 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.348412 14706 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:26.350021 14706 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.350489 14706 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.350617 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.351678 14710 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:26.352270 14710 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.352771 14710 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:26.270002 14339 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.349256 14569 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.349246 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:26.360009 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.361156 10513 raft_consensus.cc:2410] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:26.362550 14569 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.285409 14681 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.213924 10331 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 2 to 7, term changed from 7 to 10, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:26.367750 10331 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 6, term changed from 1 to 8, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to <none>, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 8 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:11:26.376052 14549 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.624s	user 0.012s	sys 0.001s
I20260709 12:11:26.377517 10429 raft_consensus.cc:2410] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:26.282675 14677 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 12:11:26.299823 14599 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.381773 14650 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:26.382651 14650 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:26.384649 14466 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.384052 14650 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.388005 10505 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:26.303819 14667 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.249989 14746 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.308799 14673 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.395960 14673 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:26.408056 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.298478 10511 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.429080 10511 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.284559 14557 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.464421 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.465334 10436 raft_consensus.cc:2393] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 3: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:26.415423 14460 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:26.478202 10553 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.130s (SENT)
I20260709 12:11:26.479950 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.462574 14715 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.495100 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.496341 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.444813 14566 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:11:26.493997 14541 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:26.380190 14599 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.514420 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.463801 14557 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.503042 14460 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.501915 14605 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:26.502238 14558 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:26.512447 14562 consensus_meta.cc:287] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.718s	user 0.016s	sys 0.006s
I20260709 12:11:26.537679 14599 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:26.543224 14721 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12.
W20260709 12:11:26.570154 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 156395us
W20260709 12:11:26.545256 14609 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.008s
I20260709 12:11:26.547883 14557 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.515126 14541 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.548444 14667 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:26.550275 14754 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:11:26.551697 14605 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.558712 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.584538 11216 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:26.530216 14681 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.554603 14747 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:26.572307 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.565752 14559 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:26.504822 14715 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.580148 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:26.566581 14558 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:26.589224 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.577463 14609 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:26.563647 14460 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.605690 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:26.602564 14605 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:26.572736 10437 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:26.591753 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:26.583925 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:26.602460 11221 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:26.626876 10507 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:26.602237 14667 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.576390 14541 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.627519 11219 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:26.630960 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.601655 14747 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:26.618249 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:26.620241 10512 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:26.593771 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.621568 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.466759 10331 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:11:26.595574 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.600809 10511 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:11:26.585508 14562 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:26.576071 14599 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.622076 14558 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.671743 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:26.614956 14466 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.695632 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.669879 10437 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:26.670149 10511 pending_rounds.cc:85] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 6
I20260709 12:11:26.699270 10511 pending_rounds.cc:107] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:11:26.699630 10511 raft_consensus.cc:2889] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:26.700784 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2375) took 1301 ms. Trace:
I20260709 12:11:26.674530 10432 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:26.702162 11213 raft_consensus.cc:2393] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 3: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:26.619504 14681 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:26.676646 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.705960 10430 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.673007 14667 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.574363 14665 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.676577 14562 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.681478 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:26.618955 14559 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.673712 14599 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:26.691650 10433 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2451) took 1968 ms (client timeout 1620 ms). Trace:
W20260709 12:11:26.703270 14598 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.589s	user 0.014s	sys 0.004s
I20260709 12:11:26.630570 14730 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.701360 10511 rpcz_store.cc:276] 0709 12:11:25.399518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.399704 (+   186us) service_pool.cc:224] Handling call
0709 12:11:25.400090 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:26.296586 (+896496us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b54002db2a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a56dc0 00007f2c92ae11c0 00007f2c92ae2231 00007f2c92aded8f 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c
0709 12:11:26.700440 (+403854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.700704 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91138944}
I20260709 12:11:26.702193 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.831094 10435 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:26.824551 10433 rpcz_store.cc:269] 0709 12:11:24.723350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.743923 (+ 20573us) service_pool.cc:224] Handling call
0709 12:11:26.691340 (+1947417us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540033dbd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:26.691505 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175409536}
I20260709 12:11:26.709571 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:26.701370 10513 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:26.814394 10504 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:26.689056 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:26.820755 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:26.821398 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92053.7us
W20260709 12:11:26.703590 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 291631us
W20260709 12:11:26.700680 14213 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:11:26.836212 11178 leader_election.cc:400] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
W20260709 12:11:26.836916 10478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.656s (SENT) [suppressed 44 similar messages]
I20260709 12:11:26.632005 14648 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:11:26.859439 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.859676 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.868057 14580 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:26.870527 14711 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.303s	user 0.003s	sys 0.005s
I20260709 12:11:26.872434 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 1997) took 1964 ms. Trace:
I20260709 12:11:26.872852 10506 rpcz_store.cc:276] 0709 12:11:24.908320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.908494 (+   174us) service_pool.cc:224] Handling call
0709 12:11:24.952391 (+ 43897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:24.967791 (+ 15400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:24.967825 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:24.982321 (+ 14496us) log.cc:844] Serialized 26 byte log entry
0709 12:11:25.291302 (+308981us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:25.291367 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:25.291482 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:26.675082 (+1383600us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:26.872062 (+196980us) raft_consensus.cc:1685] finished
0709 12:11:26.872092 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:26.872356 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16431,"raft.queue_time_us":15522,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":8336,"spinlock_wait_cycles":12993280,"thread_start_us":273419,"threads_started":1,"wal-append.queue_time_us":488084}
I20260709 12:11:26.880295 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.881153 10436 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:26.882215 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.883306 14704 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:26.883960 14704 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.885411 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1909) took 1397 ms. Trace:
I20260709 12:11:26.885931 10429 rpcz_store.cc:276] 0709 12:11:25.487654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.584573 (+ 96919us) service_pool.cc:224] Handling call
0709 12:11:26.885122 (+1300549us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400166cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:26.885322 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721814784}
I20260709 12:11:26.882817 10436 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.904155 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:26.907598 10434 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:26.812018 14651 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:26.909986 11212 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:26.686913 14534 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
W20260709 12:11:26.856699 14666 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.051s	user 0.022s	sys 0.000s
I20260709 12:11:26.923502 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:26.926707 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 305559us
W20260709 12:11:26.905359 14754 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:11:26.934552 10431 raft_consensus.cc:2410] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:11:26.860368 14598 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:26.886713 14704 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.862920 14681 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:26.945230 10436 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:26.947403 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:26.968611 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:26.969537 10438 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:26.970163 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1931) took 1046 ms. Trace:
I20260709 12:11:26.970790 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:26.970513 10438 rpcz_store.cc:276] 0709 12:11:25.923875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.923526 (+999651us) service_pool.cc:224] Handling call
0709 12:11:26.970091 (+ 46565us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:26.973623 10438 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:11:26.974037 10438 pending_rounds.cc:85] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 6
I20260709 12:11:26.974329 10438 pending_rounds.cc:107] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:11:26.980515 14599 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.981206 11213 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:11:26.981257 14599 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:11:26.983014 14559 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:26.984390 14599 raft_consensus.cc:2764] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:11:26.986033 14562 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.988013 14673 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:26.989085 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:26.989771 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.991072 14715 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:26.968755 14747 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:26.993932 14747 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.996099 14460 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:26.997466 14672 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.786s	user 0.015s	sys 0.000s
I20260709 12:11:26.998081 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2476) took 1039 ms. Trace:
I20260709 12:11:26.997999 14672 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.998546 10434 rpcz_store.cc:276] 0709 12:11:25.958959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.688729 (+729770us) service_pool.cc:224] Handling call
0709 12:11:26.997842 (+309113us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b54003962d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:26.997970 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170264960}
I20260709 12:11:26.859948 14730 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.002489 10433 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 7.
I20260709 12:11:27.003159 14450 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:27.003950 14730 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.004940 14746 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:27.005581 14746 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.006245 14666 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:27.006708 14666 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:27.007733 14666 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.008908 10431 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:11:27.009146 14666 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:27.009379 10431 pending_rounds.cc:85] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 6
I20260709 12:11:27.009670 10431 pending_rounds.cc:107] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:11:27.009917 10431 raft_consensus.cc:2889] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:27.010671 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1924) took 1518 ms. Trace:
I20260709 12:11:27.012687 10583 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.012619 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.013248 14239 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:27.013634 10586 raft_consensus.cc:2410] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:11:27.017544 14392 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:11:27.018102 14756 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.480s	user 0.004s	sys 0.010s
I20260709 12:11:26.921598 14580 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.021199 10429 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:27.022387 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.025183 14665 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:27.025538 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:26.974834 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 784091us
I20260709 12:11:27.003798 14450 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:26.974619 10438 raft_consensus.cc:2889] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:26.978605 14534 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:11:26.947834 14640 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.008s
I20260709 12:11:26.937143 14651 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:27.030942 10429 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:27.031850 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.030616 10431 rpcz_store.cc:276] 0709 12:11:25.492041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.008032 (+1515991us) service_pool.cc:224] Handling call
0709 12:11:27.008409 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.010390 (+  1981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.010607 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.040916 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.042351 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 429) took 1004 ms. Trace:
I20260709 12:11:27.054682 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.034861 10586 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:11:27.036222 10578 raft_consensus.cc:606] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:27.033087 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.058722 10578 raft_consensus.cc:614] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:27.044660 10583 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.035547 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.033695 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.034549 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.047211 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.037705 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1921) took 1546 ms. Trace:
I20260709 12:11:27.061417 10438 rpcz_store.cc:276] 0709 12:11:25.491253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.972681 (+1481428us) service_pool.cc:224] Handling call
0709 12:11:26.973165 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.036957 (+ 63792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.037491 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.036296 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.063462 10436 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:27.065960 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.066300 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.066673 10510 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:27.067097 10586 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.044008 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.068189 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.065979 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.068612 10587 raft_consensus.cc:2393] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.069144 10582 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:27.031752 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:27.073926 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.073983 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.078377 10586 raft_consensus.cc:2410] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:27.074354 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.054590 10509 rpcz_store.cc:276] 0709 12:11:26.038412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.039229 (+   817us) service_pool.cc:224] Handling call
0709 12:11:26.039664 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:26.040147 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:26.040188 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:26.054171 (+ 13983us) log.cc:844] Serialized 26 byte log entry
0709 12:11:26.181828 (+127657us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:26.271143 (+ 89315us) log.cc:844] Serialized 14 byte log entry
0709 12:11:26.273409 (+  2266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.273527 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:26.980974 (+707447us) spinlock_profiling.cc:243] Waited 7.19 ms on lock 0x7b18002de9e0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:27.041954 (+ 60980us) raft_consensus.cc:1685] finished
0709 12:11:27.042077 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.042262 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1857,"prepare.queue_time_us":89107,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":5795,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":10971,"spinlock_wait_cycles":16529792,"thread_start_us":99783,"threads_started":1,"wal-append.queue_time_us":212249}
I20260709 12:11:27.065079 10430 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:27.081084 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2477) took 1120 ms. Trace:
I20260709 12:11:27.081857 10436 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:27.082949 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1928) took 1160 ms. Trace:
W20260709 12:11:27.034972 14706 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.684s	user 0.019s	sys 0.024s
I20260709 12:11:27.100801 10431 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.101814 10585 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:27.103397 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.105813 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.107431 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.108757 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.081669 10430 rpcz_store.cc:276] 0709 12:11:25.960657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.625142 (+664485us) service_pool.cc:224] Handling call
0709 12:11:27.080983 (+455841us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.111443 10430 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.111460 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 445) took 1190 ms. Trace:
I20260709 12:11:27.112155 10431 rpcz_store.cc:276] 0709 12:11:25.921292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.075105 (+1153813us) service_pool.cc:224] Handling call
0709 12:11:27.085035 (+  9930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.111084 (+ 26049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.111376 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.084802 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.085469 10584 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.068960 10579 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:27.112159 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1930) took 1188 ms. Trace:
I20260709 12:11:27.124785 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2474) took 1171 ms. Trace:
I20260709 12:11:27.067649 14559 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.083307 10436 rpcz_store.cc:276] 0709 12:11:25.922813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.080873 (+1158060us) service_pool.cc:224] Handling call
0709 12:11:27.081367 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.082659 (+  1292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.082887 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.090201 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.090281 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.090803 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.091480 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.124001 10430 rpcz_store.cc:276] 0709 12:11:25.923586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.110875 (+1187289us) service_pool.cc:224] Handling call
0709 12:11:27.111244 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.111890 (+   646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.112100 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:27.085741 14710 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.732s	user 0.018s	sys 0.020s
I20260709 12:11:27.027156 14681 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.135659 10433 rpcz_store.cc:276] 0709 12:11:25.953304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.859341 (+906037us) service_pool.cc:224] Handling call
0709 12:11:27.124512 (+265171us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400320354. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:27.124680 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323878656}
I20260709 12:11:27.137480 10433 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:27.121140 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.102384 14677 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:27.131531 10583 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.132084 10580 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:27.067966 14579 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:11:27.132479 10511 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:27.086197 10510 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:27.092644 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.133929 10436 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:27.154838 14460 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:27.157018 14460 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.157567 10511 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:27.048683 14598 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.158600 14598 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.159093 14681 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.159149 10509 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:27.160634 14580 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:27.139438 14710 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.127619 14392 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:27.109550 10507 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:27.166999 10509 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:27.163647 14646 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.169180 14646 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.144600 14566 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:27.171379 14566 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:27.172575 14646 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.172420 14566 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.173740 14541 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.103571 14665 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.176246 14665 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.132370 10438 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:11:27.116698 14706 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.146018 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:27.180205 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:27.103456 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2472) took 1152 ms. Trace:
I20260709 12:11:27.187134 10432 rpcz_store.cc:276] 0709 12:11:25.950777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.530847 (+580070us) service_pool.cc:224] Handling call
0709 12:11:27.103232 (+572385us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b5400437954. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:27.103381 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166531456}
I20260709 12:11:27.188366 10506 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:27.188526 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.189293 10432 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:27.190265 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.044009 14640 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.193096 14640 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.187144 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:27.194468 10505 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:27.179006 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.178874 10438 pending_rounds.cc:85] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 3
I20260709 12:11:27.196061 10438 pending_rounds.cc:107] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:11:27.196346 10438 raft_consensus.cc:2889] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:27.073985 14518 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:11:27.035823 14704 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:11:27.155435 10510 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.156384 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1935) took 1231 ms. Trace:
I20260709 12:11:27.084398 14609 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.198870 10432 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.197775 10436 rpcz_store.cc:276] 0709 12:11:25.925425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.133258 (+1207833us) service_pool.cc:224] Handling call
0709 12:11:27.133575 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.156084 (+ 22509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.156301 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.143929 14677 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.139776 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1936) took 1214 ms. Trace:
I20260709 12:11:27.175468  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:11:27.207563 10510 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.178553 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.208655 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.180987 14706 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.212998 14569 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:11:27.181783 14646 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:27.198248 14609 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:27.215485 14605 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.541s	user 0.014s	sys 0.000s
I20260709 12:11:27.215970 14605 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.217391 14557 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:27.217921 14609 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:27.143746 14730 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.218760 14730 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.208804 14677 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:27.219944 14541 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.220927 14747 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.200815 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.222165 10436 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:27.221714 14747 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.223570 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.224524 10436 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:27.225695 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.226114 14747 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:27.226513 10436 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:27.146510 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1933) took 1222 ms. Trace:
I20260709 12:11:27.142000 10435 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:27.240808 14667 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:27.241309 10435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 441) took 1781 ms (client timeout 1769 ms). Trace:
W20260709 12:11:27.241767 14558 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
W20260709 12:11:27.241920 10435 rpcz_store.cc:269] 0709 12:11:25.459854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.701878 (+1242024us) service_pool.cc:224] Handling call
0709 12:11:27.241190 (+539312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.178707 14665 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.243582 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:27.244463 10435 raft_consensus.cc:2376] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13. Re-sending same reply.
I20260709 12:11:27.209087 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:27.209134 10434 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.210948 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.246758 10429 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:27.248282 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.249922 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.210593 10433 rpcz_store.cc:276] 0709 12:11:25.925716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.136706 (+1210990us) service_pool.cc:224] Handling call
0709 12:11:27.137143 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.138911 (+  1768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.139648 (+   737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.251413 10433 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:27.209872 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1927) took 1287 ms. Trace:
I20260709 12:11:27.251977 10438 rpcz_store.cc:276] 0709 12:11:25.922510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.078761 (+1156251us) service_pool.cc:224] Handling call
0709 12:11:27.122765 (+ 44004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.196876 (+ 74111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.209796 (+ 12920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.252936 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2479) took 1289 ms. Trace:
I20260709 12:11:27.181691 10437 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:27.218149 14605 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.228175 10506 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:27.228739 14681 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:27.176745 10509 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:27.167874 14691 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.260267 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:27.221704 14730 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.174705 14460 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.236384 14746 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.207618  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:27.253870 10433 rpcz_store.cc:276] 0709 12:11:25.962308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.250909 (+1288601us) service_pool.cc:224] Handling call
0709 12:11:27.251232 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.251790 (+   558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.251998 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.240535 10430 rpcz_store.cc:276] 0709 12:11:25.924461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.131884 (+1207423us) service_pool.cc:224] Handling call
0709 12:11:27.132228 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.146152 (+ 13924us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:27.146184 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.146227 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.146262 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.146334 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.146448 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13511,"raft.queue_time_us":14090,"spinlock_wait_cycles":23608704}
I20260709 12:11:27.139254 14559 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.299186 10579 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.300203 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 448) took 1206 ms. Trace:
I20260709 12:11:27.300688 10433 rpcz_store.cc:276] 0709 12:11:26.094006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.293433 (+1199427us) service_pool.cc:224] Handling call
0709 12:11:27.293814 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.299877 (+  6063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.299913 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.299955 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:27.299982 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.300045 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.300147 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5621}
I20260709 12:11:27.257208 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.242420 14558 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.259330 14667 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:27.307680 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.228833 14710 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.260564 14721 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:27.237142 14666 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:27.261143 10432 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.261855 14681 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.263536 14691 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.265443 14450 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.266546 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2481) took 1225 ms. Trace:
I20260709 12:11:27.203809 14651 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:27.262715 14767 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:11:27.270090 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:27.388573 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.388897 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.271601 10583 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.272985 10578 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:27.273563 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 302856us
I20260709 12:11:27.393977 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:27.386420 14769 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:27.284929 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.285030 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.285352 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.400195 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.286567 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:27.400950 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.260360 14708 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:27.303138 11178 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.300235 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.401710 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:27.402177 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104008us
I20260709 12:11:27.402437 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.403476 10578 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.403388 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:27.403947 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.404556 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.259208 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:27.320308 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.319490 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.405757 11219 raft_consensus.cc:1813] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Handling vote request from an unknown peer 33d77e3063684586a737bccf358786d8
I20260709 12:11:27.326782 10579 raft_consensus.cc:2410] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:11:27.327443 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.406971 10587 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:27.269070 14569 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:27.413542 14569 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.414075 14569 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:27.390357 14605 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.416550 14605 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.260716 14579 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.418541 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.419214 14579 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.420048 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:27.331862 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:27.421363 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:27.421504 10511 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.422158 10506 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:27.326037 14667 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.423237 14667 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.333422 14558 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:27.423769 14761 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.280s	user 0.000s	sys 0.003s
I20260709 12:11:27.426209 14666 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.266600 14672 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.427232 14648 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:27.427350 14746 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:27.427727 14648 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:27.427803 14598 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.428201 14648 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.212934 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.428925 14746 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:11:27.429165 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 216528us
I20260709 12:11:27.429755 14769 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.430414 14746 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:27.432721 14691 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.433390 14691 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:27.434219 14691 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.297544 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.350201 14681 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:27.409076 10581 raft_consensus.cc:2410] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:11:27.397853 14213 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.409418 11219 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:27.295361 14747 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:11:27.419196 14708 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.268886 14683 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:11:27.298899 14650 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:11:27.477837 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.478888 10579 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:27.480227 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.481738 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.409883 14677 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.408010 10583 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:27.437487 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:27.487732 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.444278 10584 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.488582 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.490082 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.426673 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2483) took 1324 ms. Trace:
I20260709 12:11:27.490697 10583 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:27.491199 10436 rpcz_store.cc:276] 0709 12:11:26.102724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.425554 (+1322830us) service_pool.cc:224] Handling call
0709 12:11:27.425934 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.426183 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.426220 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.426278 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.426339 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.426422 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.426584 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.493659 10436 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:27.494503 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2485) took 1236 ms. Trace:
I20260709 12:11:27.494923 10436 rpcz_store.cc:276] 0709 12:11:26.258383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.492791 (+1234408us) service_pool.cc:224] Handling call
0709 12:11:27.493266 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.494172 (+   906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.494429 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.496649 10436 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.295914 14730 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:27.319464 14665 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.093s
W20260709 12:11:27.448211 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44813.8us
I20260709 12:11:27.474124 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.482517 10579 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:27.515322 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:27.515545 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 78158.9us
W20260709 12:11:27.518239 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.521191 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.526705 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.517601 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.530301 10584 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:27.531591 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.533087 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.534540 10584 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.441485 14239 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:27.454659 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:27.436662 14766 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.241s	user 0.003s	sys 0.000s
I20260709 12:11:27.430572 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.437685 14770 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:27.546139 14770 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.354936 14721 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.499315 14772 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.499470 14559 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.499521 14720 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:27.438014 14672 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.472787 14534 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:11:27.501492 14708 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.501739 14562 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:27.488507 10587 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:27.320400 14518 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 12:11:27.435804 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1922) took 1944 ms. Trace:
I20260709 12:11:27.588826 10434 rpcz_store.cc:276] 0709 12:11:25.491519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.999567 (+1508048us) service_pool.cc:224] Handling call
0709 12:11:27.208913 (+209346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.246667 (+ 37754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.435597 (+188930us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400436f50. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:27.435734 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480519936}
I20260709 12:11:27.445181 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.543406 14239 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.509300 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1939) took 1237 ms. Trace:
I20260709 12:11:27.605253 10436 rpcz_store.cc:276] 0709 12:11:26.271560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.496029 (+1224469us) service_pool.cc:224] Handling call
0709 12:11:27.496407 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.497147 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.509213 (+ 12066us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:27.510269 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 487936us
I20260709 12:11:27.609330 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.508633 10511 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:27.611069 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.613044 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.386850 10437 rpcz_store.cc:276] 0709 12:11:26.040920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.714773 (+673853us) service_pool.cc:224] Handling call
0709 12:11:27.266302 (+551529us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b540015f4d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:27.266469 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584510336}
I20260709 12:11:27.613914 10586 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:27.346864 14646 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:27.511554 10432 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:27.615162 10433 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.615936 10432 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:27.616854 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1944) took 1311 ms. Trace:
I20260709 12:11:27.617391 10432 rpcz_store.cc:276] 0709 12:11:26.305758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.615198 (+1309440us) service_pool.cc:224] Handling call
0709 12:11:27.615659 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.616548 (+   889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.616772 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.485478 10580 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:27.537036 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.619551 10581 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.619741 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.620373 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.538281 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.621717 10585 raft_consensus.cc:2376] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13. Re-sending same reply.
I20260709 12:11:27.622658 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.623101 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.503935 14730 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.542264 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.434667 14691 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.542157 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.636154 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.426842 14666 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:27.522710 14580 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:27.550212 14721 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.438740 14213 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.536358 14665 consensus_queue.cc:1243] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:11:27.444540 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:27.584643 14773 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:27.627355 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:27.628850 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.673501 10585 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:27.617588 14460 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.629568 10432 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.629576 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:27.677134 10587 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:27.677238 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 451) took 1368 ms. Trace:
I20260709 12:11:27.620566 10583 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:27.677814 10432 rpcz_store.cc:276] 0709 12:11:26.309318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.618594 (+1309276us) service_pool.cc:224] Handling call
0709 12:11:27.629301 (+ 10707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.676841 (+ 47540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.677141 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.583087 14715 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.582615 14683 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:27.582681 14720 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:27.585202 14562 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:27.618541 14605 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:27.671404 14770 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.667809 10506 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:27.539602 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:27.625133 14598 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:27.684309 10580 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:11:27.685418 10583 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:27.582790 14672 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.635630 14730 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:27.687927 10587 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:27.637533 10437 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.688334 10509 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
W20260709 12:11:27.708935 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 169418us
I20260709 12:11:27.709156 10583 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:27.580231 14772 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.713933 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:27.732386 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.719385 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2488) took 1391 ms. Trace:
I20260709 12:11:27.737617 10437 rpcz_store.cc:276] 0709 12:11:26.328082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.636760 (+1308678us) service_pool.cc:224] Handling call
0709 12:11:27.637291 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.718965 (+ 81674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.719291 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.740540 11221 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:27.741984 11217 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:27.694180 14683 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.748831 14213 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.717255 14580 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:27.750312 14683 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.751029 14580 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.751415 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:27.617151 14239 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:27.752154 14677 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.688189 14460 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.649209 14646 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:27.753577 14579 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:27.756974 14672 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:27.766103 14681 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.781075 14666 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:27.782228 14720 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.783995 10404 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:27.834954 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51140.7us
I20260709 12:11:27.785523 14534 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:11:27.837414 14672 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:27.837492 14534 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:27.838385 14672 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:27.688913 14715 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.665885 10581 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.689656 14721 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.746379 14569 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.851315 10581 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:11:27.815290 10586 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:27.820096 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 85272.1us
I20260709 12:11:27.827009 10579 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 7. Current term is 8. Ops: []
W20260709 12:11:27.829564 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 424077us
W20260709 12:11:27.832883 14774 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:11:27.786437 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 483437us
I20260709 12:11:27.803308 14580 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:27.877432 13946 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.735795 10583 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:27.855832 14780 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 12:11:27.854880 10434 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:27.855024 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:27.855581 10506 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:27.855686 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.842350 10437 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:27.885442 10552 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:27.883975 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 255046us
I20260709 12:11:27.848007 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1942) took 1543 ms. Trace:
I20260709 12:11:27.886277 10436 rpcz_store.cc:276] 0709 12:11:26.304882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.606481 (+1301599us) service_pool.cc:224] Handling call
0709 12:11:27.607030 (+   549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.607349 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:27.624792 (+ 17443us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.846549 (+221757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.846612 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.846659 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.846727 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.847924 (+  1197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67831,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":120,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6198,"thread_start_us":221232,"threads_started":1}
I20260709 12:11:27.887431 10437 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.890831 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.896234 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.848017 14648 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.849849 14721 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.927384 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:27.853098 14667 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:27.935608 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:27.937389 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.898022 14534 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:27.943610 10582 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:27.899159 14666 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:27.899981 14640 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:27.898415 14683 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:27.945685 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50677.2us
I20260709 12:11:27.901207 14450 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:27.884076 14650 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:11:27.891124 13946 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:27.954178 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:27.971979 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:27.893105 14747 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:11:27.891629 14559 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:11:27.979722 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:27.947683 14566 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.009s	sys 0.011s
I20260709 12:11:27.906772 14392 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:11:27.997907 14423 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:27.951185 14640 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:27.957487 14691 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:27.997196 14566 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.010s	sys 0.011s
I20260709 12:11:27.977298 14534 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.971258 14677 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.995664 14683 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:28.054039 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 623504us
I20260709 12:11:28.087745 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:28.026188 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:28.089112 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 63031.3us
I20260709 12:11:28.091173 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:28.092546 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:28.090883 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 188469us
I20260709 12:11:28.026361 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:28.034937 14771 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:28.038004 10583 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:28.012857 14640 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:28.054634 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 374682us
W20260709 12:11:28.053614 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:28.051251 14776 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:11:28.098033 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:28.022675 14706 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:11:28.048623 14785 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 12:11:28.047849 14784 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:11:28.052794 14715 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:28.110561 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:28.153090 10506 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:28.111275 10583 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:11:28.154274 10583 pending_rounds.cc:85] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:11:28.154593 10583 pending_rounds.cc:107] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:11:28.154870 10583 raft_consensus.cc:2889] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:28.111558 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:28.112501 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:28.055348 14598 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:28.116894 10509 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:28.157625 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:28.008610 14787 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:28.035444 10463 heartbeater.cc:646] Failed to heartbeat to 127.9.11.126:42677 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.11.126:42677 timed out after 15.000s (SENT)
I20260709 12:11:28.123740 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:28.103487 14672 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:28.097470 14771 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:28.054203 14558 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:28.124670 14646 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:11:28.139695 14772 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:28.040504 14549 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.144668 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:28.189231 10510 raft_consensus.cc:2410] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:11:28.044484 14790 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:11:28.149631 10505 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:11:28.159221 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 183771us
I20260709 12:11:28.167860 14715 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:28.158344 10512 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:28.196686 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 526) took 1163 ms. Trace:
W20260709 12:11:28.124274 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.118103 14213 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:28.078645 14666 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.052126 14681 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:28.059633 14562 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.184391 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:28.158526 14777 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 12:11:28.093413 14677 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:28.172988 14515 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:11:28.174633 14558 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:28.165644 14598 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:28.148247 14706 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:28.126084 14781 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.273s	user 0.002s	sys 0.000s
I20260709 12:11:28.174628 14605 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:28.210155 14648 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:28.197289 11212 rpcz_store.cc:276] 0709 12:11:27.028083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.028280 (+   197us) service_pool.cc:224] Handling call
0709 12:11:27.028680 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.035869 (+  7189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.035903 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.036458 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.139205 (+102747us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18003af3c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:27.607211 (+468006us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.607255 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.607338 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:28.191479 (+584141us) raft_consensus.cc:1685] finished
0709 12:11:28.191513 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:28.191823 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6740,"raft.queue_time_us":56602,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":10148,"spinlock_wait_cycles":42139392,"thread_start_us":467772,"threads_started":1,"wal-append.queue_time_us":687200}
I20260709 12:11:28.183485 14691 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.182682 14683 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:28.185307 14569 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:28.218853 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 199955us
I20260709 12:11:28.268275 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2478) took 2306 ms. Trace:
I20260709 12:11:28.268837 10429 rpcz_store.cc:276] 0709 12:11:25.961488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.248078 (+1286590us) service_pool.cc:224] Handling call
0709 12:11:27.249041 (+   963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.253575 (+  4534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.253614 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.259839 (+  6225us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.667065 (+407226us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:27.667117 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.667195 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:28.267893 (+600698us) raft_consensus.cc:1685] finished
0709 12:11:28.267948 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:28.268179 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3743,"raft.queue_time_us":9938,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":158,"thread_start_us":333615,"threads_started":1,"wal-append.queue_time_us":561986}
W20260709 12:11:28.221611 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.205075  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:28.201797 14787 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:11:28.270968 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:28.272382 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.215245 14213 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:28.222796 14558 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:28.194712 14549 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.301994 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2026) took 1114 ms. Trace:
I20260709 12:11:28.302609 10508 rpcz_store.cc:276] 0709 12:11:27.187492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.189481 (+  1989us) service_pool.cc:224] Handling call
0709 12:11:27.189820 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.190069 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.190098 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.190666 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.536452 (+345786us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.536518 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.537163 (+   645us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:28.301616 (+764453us) raft_consensus.cc:1685] finished
0709 12:11:28.301651 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:28.301895 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":345400,"threads_started":1,"wal-append.queue_time_us":514367}
I20260709 12:11:28.212280 14801 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:28.291217 14796 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:11:28.291042 14808 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:28.292408 14794 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:11:28.292141 14815 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:28.260304 14605 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:28.214937 14559 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 12:11:28.335984 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 124192us
W20260709 12:11:28.336045 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 325697us
I20260709 12:11:28.336898 10436 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:28.275104 14721 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:28.312769 13946 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:28.318996 14816 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:28.208292 14795 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:11:28.269342 14667 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:28.333645 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2480) took 2294 ms. Trace:
I20260709 12:11:28.368764 10438 rpcz_store.cc:276] 0709 12:11:26.038907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.252936 (+1214029us) service_pool.cc:224] Handling call
0709 12:11:27.291559 (+ 38623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.293869 (+  2310us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:27.325748 (+ 31879us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.717761 (+392013us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:28.315236 (+597475us) spinlock_profiling.cc:243] Waited 7.47 ms on lock 0x7b2c00000f48. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:28.315816 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:11:28.316009 (+   193us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:28.316060 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.316170 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:28.321207 (+  5037us) raft_consensus.cc:1685] finished
0709 12:11:28.333369 (+ 12162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:28.333570 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2633,"prepare.queue_time_us":31683,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":4075,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":58066,"spinlock_wait_cycles":17190656,"thread_start_us":318537,"threads_started":1,"wal-append.queue_time_us":535712}
I20260709 12:11:28.224187 14562 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:28.303673 14683 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:28.339840 11212 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:28.321473 14801 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.210417 14771 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:28.348232 14803 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:11:28.349256 14793 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:11:28.356878 14807 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:28.358294 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:28.335151 14646 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 12:11:28.280285 14691 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.336019 14605 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:28.483036 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:28.212854 14666 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.367960 14816 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:28.483790 10583 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
W20260709 12:11:28.452342 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 94180.2us
I20260709 12:11:28.222517 14772 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:11:28.454435 14821 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:28.488116 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 390165us
I20260709 12:11:28.489694 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:28.454799 14549 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:28.424546 14213 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.455315 14747 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:28.456972 14677 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:28.463730 14721 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.466336 14801 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:28.467463 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1940) took 2169 ms. Trace:
I20260709 12:11:28.587294 10434 rpcz_store.cc:276] 0709 12:11:26.297658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.590144 (+1292486us) service_pool.cc:224] Handling call
0709 12:11:27.590607 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.884729 (+294122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.467167 (+582438us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54003962d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:28.467387 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600621952}
W20260709 12:11:28.588793 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 79729.2us
I20260709 12:11:28.589248 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:28.473265 14769 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:28.477073 14792 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.474s	user 0.002s	sys 0.000s
I20260709 12:11:28.448890 14805 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:28.477596 14558 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.478255 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:28.601864 10438 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:28.480594 14559 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:28.491040 11213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.006s	sys 0.005s
W20260709 12:11:28.609694 11213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.006s	sys 0.005s
I20260709 12:11:28.492937 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:28.489080 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:28.498046 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:28.477591 14771 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:28.506647 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:28.619706 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 450) took 2525 ms. Trace:
I20260709 12:11:28.620247 10433 rpcz_store.cc:276] 0709 12:11:26.094673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.302039 (+1207366us) service_pool.cc:224] Handling call
0709 12:11:27.614886 (+312847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.615633 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.619466 (+1003833us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b540034a150. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:28.619637 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718529920}
I20260709 12:11:28.621872 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:28.625025 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2413) took 1310 ms. Trace:
I20260709 12:11:28.625527 10582 rpcz_store.cc:276] 0709 12:11:27.314504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.454273 (+139769us) service_pool.cc:224] Handling call
0709 12:11:28.624706 (+1170433us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54001d60d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:28.624934 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":913872768}
W20260709 12:11:28.492611 14580 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.605s	user 0.014s	sys 0.000s
I20260709 12:11:28.516659 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1932) took 2592 ms. Trace:
I20260709 12:11:28.500062 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:28.484242 14816 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:28.607681 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:28.632216 10584 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:28.495299 14648 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:11:28.477085 14683 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.498785 14821 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.497256 14706 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.587149 14770 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:28.766446 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 260707us
I20260709 12:11:28.671826 10438 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
W20260709 12:11:28.724951 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.779798 10431 rpcz_store.cc:276] 0709 12:11:25.924179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.113520 (+1189341us) service_pool.cc:224] Handling call
0709 12:11:27.119300 (+  5780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.145277 (+ 25977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.145313 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.145879 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.544258 (+398379us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:27.604741 (+ 60483us) log.cc:844] Serialized 14 byte log entry
0709 12:11:28.076626 (+471885us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b1800446c80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c92ad7060 00007f2c92ae9716 00007f2c92afa287 00007f2c92afa208
0709 12:11:28.516099 (+439473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.516218 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:28.516281 (+    63us) raft_consensus.cc:1685] finished
0709 12:11:28.516384 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:28.516568 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28955,"prepare.queue_time_us":44267,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":4686,"raft.queue_time_us":29194,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":20561,"spinlock_wait_cycles":57318272,"thread_start_us":827551,"threads_started":2,"wal-append.queue_time_us":563944}
I20260709 12:11:28.782096 10587 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:28.622644 10433 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:28.859671 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:28.860373 10438 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:28.586402 14667 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:28.720948 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.101s	sys 0.125s
I20260709 12:11:28.798292 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:28.599730 14789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.012s	sys 0.002s
W20260709 12:11:28.861382 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.108s	sys 0.136s
W20260709 12:11:28.862300 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 769787us
I20260709 12:11:28.863324 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:28.596891 14747 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:28.868456 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:28.869900 10434 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:28.617544 13946 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:28.717204 14812 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.885201 10434 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:28.888078 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:28.684098 14824 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.206s
I20260709 12:11:28.658417 14769 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:28.594192 14811 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.371s
W20260709 12:11:28.881655 14789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.012s	sys 0.027s
I20260709 12:11:28.600487 14810 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.389s
I20260709 12:11:28.605893 14721 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.889500 13946 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:28.928509 14804 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:28.861357 14773 consensus_meta.cc:287] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.173s	user 0.015s	sys 0.003s
I20260709 12:11:28.935519 10438 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:11:28.857522 14580 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.634397 14818 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.318s
I20260709 12:11:28.909513 14569 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.636252 14559 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.609066 14805 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.977303 10438 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:11:28.939086 14534 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.851s	user 0.014s	sys 0.001s
I20260709 12:11:28.985571 10437 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:28.958714 14801 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:11:28.908144 14812 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.963686 14773 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:28.889367 14667 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:28.969239 14710 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:11:29.010134 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 521140us
I20260709 12:11:28.937042 14769 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:28.962026 14789 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:29.016163 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 518175us
I20260709 12:11:29.013269 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:29.018257 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:29.021390 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:28.987339 14534 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:29.013824 14720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.022s	sys 0.032s
I20260709 12:11:29.009268 14691 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:28.969929 14804 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:29.027626 14640 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.887s	user 0.015s	sys 0.001s
I20260709 12:11:29.018942 14666 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:28.980944 14569 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.024420 14710 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:29.065043 14559 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.067538 14569 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.075536 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:29.076665 11221 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:29.033942 14558 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.078500 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:29.020181 14789 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:29.126703 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 263480us
I20260709 12:11:29.153649 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:29.160939 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 103026us
I20260709 12:11:29.175355 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:29.151678 14569 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.024688 14747 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.018623 13946 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.088585 14770 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:29.079790 14839 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:29.106154 14823 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
I20260709 12:11:29.106110 14838 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:29.052564 14640 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.187463 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:29.164036 14842 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:29.164929 14821 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:29.213277 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 195503us
I20260709 12:11:29.217372 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:29.175639 14837 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.183420 14839 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.183192 14770 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.188270 14824 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:29.187790 14843 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:29.221791 14812 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.237644 11216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 610) took 1298 ms. Trace:
I20260709 12:11:29.220976 14721 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.226914 14838 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:29.238051 11216 rpcz_store.cc:276] 0709 12:11:27.938956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.940614 (+  1658us) service_pool.cc:224] Handling call
0709 12:11:27.970969 (+ 30355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:28.224733 (+253764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:28.224769 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:28.260682 (+ 35913us) log.cc:844] Serialized 26 byte log entry
0709 12:11:28.446967 (+186285us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:28.447044 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.447152 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.237381 (+790229us) raft_consensus.cc:1685] finished
0709 12:11:29.237411 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.237577 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13473,"raft.queue_time_us":152221,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11717,"thread_start_us":403417,"threads_started":2,"wal-append.queue_time_us":318402}
I20260709 12:11:29.229707 14821 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:29.184892 14569 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.229336 14770 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:29.230829 14824 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.228873 14839 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.224100 14845 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:29.234516 14843 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.188858 14710 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.267465 14804 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.246081 14838 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:29.259179 14841 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:11:29.279335 14838 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:29.247880 14821 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:29.280192 14808 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.944s	user 0.016s	sys 0.000s
I20260709 12:11:29.280405 14821 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:29.280704 14808 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.223114 14842 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.229570 14837 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.251386 14769 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.254141 14843 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.260196 14789 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:29.277990 14771 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.000s
I20260709 12:11:29.278364 14849 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:29.273506 14816 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.283490 14849 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.284204 14837 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.252120 14845 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:29.251148 14824 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:29.276542 14730 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.492s	user 0.014s	sys 0.002s
I20260709 12:11:29.368419 14730 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.338228 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1958) took 1330 ms. Trace:
I20260709 12:11:29.220136 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:29.298215 14842 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:29.382269 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 162013us
I20260709 12:11:29.300088 14843 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.285496 14845 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.295997 14849 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.387924 10432 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
W20260709 12:11:29.361002 14239 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.423s	user 0.015s	sys 0.003s
I20260709 12:11:29.390004 14239 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.377978 10433 rpcz_store.cc:276] 0709 12:11:28.007324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.860376 (+853052us) service_pool.cc:224] Handling call
0709 12:11:28.860773 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:28.910950 (+ 50177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:28.911031 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:28.933625 (+ 22594us) log.cc:844] Serialized 26 byte log entry
0709 12:11:29.153170 (+219545us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:29.153219 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.153314 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.337152 (+183838us) raft_consensus.cc:1685] finished
0709 12:11:29.337251 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.337859 (+   608us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31033,"raft.queue_time_us":46854,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":17419,"thread_start_us":175879,"threads_started":1,"wal-append.queue_time_us":261530}
I20260709 12:11:29.371030 10429 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
W20260709 12:11:29.393678 14598 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.127s	user 0.008s	sys 0.012s
I20260709 12:11:29.251307 14667 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.375535 14580 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.390529 14640 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:29.338122 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 150822us
I20260709 12:11:29.393690 14239 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.397152 14730 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.396919 14706 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:29.382802 14842 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:29.399338 14730 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:29.399986 14842 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:29.400089 14821 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.395509 14598 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.403855 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.407631 14769 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:29.408020 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.408280 14769 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:29.409018 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.410187 10506 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:29.408831 14769 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:29.408018 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.427080 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.429495 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:29.436090 14562 consensus_meta.cc:287] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.017s	user 0.011s	sys 0.008s
I20260709 12:11:29.437006 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 455) took 2239 ms. Trace:
I20260709 12:11:29.437284 14720 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.437526 10437 rpcz_store.cc:276] 0709 12:11:27.197656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.888445 (+690789us) service_pool.cc:224] Handling call
0709 12:11:27.889122 (+   677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:28.986139 (+1097017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.436339 (+450200us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002666d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:29.436763 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2520950144}
I20260709 12:11:29.438160 14720 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.411470 14569 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.440845 14569 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:29.441855 14569 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.444695 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2523) took 2113 ms. Trace:
I20260709 12:11:29.445215 10584 rpcz_store.cc:276] 0709 12:11:27.331577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.652536 (+320959us) service_pool.cc:224] Handling call
0709 12:11:28.631935 (+979399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:28.778782 (+146847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.444103 (+665321us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54001d60d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:29.444429 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2251289344}
I20260709 12:11:29.457870 14450 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:29.458539 14450 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:29.459195 14821 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.461767 14460 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.471328 14816 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.471431 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1961) took 1463 ms. Trace:
I20260709 12:11:29.472849 10431 rpcz_store.cc:276] 0709 12:11:28.008199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.871797 (+863598us) service_pool.cc:224] Handling call
0709 12:11:28.884394 (+ 12597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:29.085868 (+201474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:29.085905 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:29.086471 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:11:29.340190 (+253719us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:29.423702 (+ 83512us) log.cc:844] Serialized 14 byte log entry
0709 12:11:29.466829 (+ 43127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.470777 (+  3948us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.470832 (+    55us) raft_consensus.cc:1685] finished
0709 12:11:29.470843 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.471166 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15806,"prepare.queue_time_us":54574,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":75359,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":11599,"thread_start_us":516079,"threads_started":3,"wal-append.queue_time_us":177112}
I20260709 12:11:29.424973 14239 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:29.468986 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:29.401746 14842 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.494585 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.503199 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1965) took 1492 ms. Trace:
W20260709 12:11:29.504465 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 92162.3us
I20260709 12:11:29.511478 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.521139 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:29.522312 10583 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.522782 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:29.524324 10587 raft_consensus.cc:2410] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:11:29.525517 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:29.525743 10584 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 12:11:29.522635 14786 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.339s	user 0.012s	sys 0.006s
I20260709 12:11:29.526762 10584 pending_rounds.cc:85] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 8
W20260709 12:11:29.533792 14791 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 1.341s	user 0.002s	sys 0.001s
I20260709 12:11:29.534436 14651 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.530390 10584 pending_rounds.cc:107] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:11:29.514284 14598 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:29.544760 10511 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:29.547859 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:29.548820 10586 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:29.552184 14800 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 1.414s	user 0.004s	sys 0.002s
I20260709 12:11:29.554171 14651 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.554868 14833 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:29.555464 14833 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:29.557646 10582 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 9: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:29.558876 14704 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:29.559350 14704 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.503754 10434 rpcz_store.cc:276] 0709 12:11:28.009343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.893724 (+884381us) service_pool.cc:224] Handling call
0709 12:11:28.929345 (+ 35621us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:28.929854 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:28.929896 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:28.937972 (+  8076us) log.cc:844] Serialized 52 byte log entry
0709 12:11:29.177685 (+239713us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:29.448324 (+270639us) log.cc:844] Serialized 14 byte log entry
0709 12:11:29.472384 (+ 24060us) log.cc:844] Serialized 14 byte log entry
0709 12:11:29.490240 (+ 17856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.495388 (+  5148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.498506 (+  3118us) raft_consensus.cc:1685] finished
0709 12:11:29.498518 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.501359 (+  2841us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10107,"prepare.queue_time_us":217013,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":127,"raft.queue_time_us":48385,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"spinlock_wait_cycles":30728192,"thread_start_us":172881,"threads_started":2,"wal-append.queue_time_us":286905}
I20260709 12:11:29.567255 14704 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:29.573165 10429 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1960) took 1565 ms (client timeout 1507 ms). Trace:
I20260709 12:11:29.570858 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 1945) took 2876 ms. Trace:
I20260709 12:11:29.542361 10584 raft_consensus.cc:2889] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:29.573756 10429 rpcz_store.cc:269] 0709 12:11:28.007903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.270584 (+262681us) service_pool.cc:224] Handling call
0709 12:11:29.572775 (+1302191us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400436a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:29.572982 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2513699712}
I20260709 12:11:29.495159 14450 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.508518 14239 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.575946 10432 rpcz_store.cc:276] 0709 12:11:26.695047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.678934 (+983887us) service_pool.cc:224] Handling call
0709 12:11:27.715637 (+ 36703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:29.388565 (+1672928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.570520 (+181955us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400320354. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:29.570783 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3844671872}
I20260709 12:11:29.301910 14838 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.586846 10511 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:29.556975 14651 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:29.619522 14651 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:29.619863 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 598430us
I20260709 12:11:29.620414 14651 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.600497 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 722) took 1121 ms. Trace:
I20260709 12:11:29.601874 10506 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:29.621520 11219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 603) took 1908 ms. Trace:
I20260709 12:11:29.622741 14821 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:29.623176 14810 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.602768 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.624186 14810 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.625425 10505 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:29.603751 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.605672 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2475) took 3649 ms. Trace:
I20260709 12:11:29.626471 10435 rpcz_store.cc:276] 0709 12:11:25.955873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.245390 (+1289517us) service_pool.cc:224] Handling call
0709 12:11:27.245886 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.261182 (+ 15296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.261223 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.295141 (+ 33918us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.879808 (+584667us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.901755 (+ 21947us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.901987 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.902079 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.496617 (+1594538us) spinlock_profiling.cc:243] Waited 7.12 ms on lock 0x7b1800418fc0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:29.532472 (+ 35855us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:29.605062 (+ 72590us) raft_consensus.cc:1685] finished
0709 12:11:29.605098 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.605374 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22336,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":71,"raft.queue_time_us":36325,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":11719,"spinlock_wait_cycles":31796352,"thread_start_us":458285,"threads_started":1,"wal-append.queue_time_us":731356}
I20260709 12:11:29.630085 14651 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.612341 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:29.613607 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:29.615254 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:29.617225 14706 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.598446 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2532) took 2258 ms. Trace:
I20260709 12:11:29.666190 10579 rpcz_store.cc:276] 0709 12:11:27.340038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.663075 (+323037us) service_pool.cc:224] Handling call
0709 12:11:27.663461 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.854355 (+190894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.597736 (+1743381us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540010c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:29.598184 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372456192}
I20260709 12:11:29.669005 10579 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:29.669260 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:29.670472 10581 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:29.600299 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.621533 10585 rpcz_store.cc:276] 0709 12:11:28.481610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.487196 (+  5586us) service_pool.cc:224] Handling call
0709 12:11:28.487638 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:28.992149 (+504511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:28.992200 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:28.992260 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:28.992306 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.599980 (+607674us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400335c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:29.600060 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.600285 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159614336}
I20260709 12:11:29.633116 14392 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:29.633136 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 449) took 3538 ms. Trace:
I20260709 12:11:29.635936 14650 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.637127 14747 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.638690 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:29.639657 14450 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:29.643157 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 256312us
I20260709 12:11:29.622040 11219 rpcz_store.cc:276] 0709 12:11:27.713122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.713358 (+   236us) service_pool.cc:224] Handling call
0709 12:11:27.713923 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:28.143724 (+429801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:28.143796 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:28.165695 (+ 21899us) log.cc:844] Serialized 26 byte log entry
0709 12:11:28.300928 (+135233us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18003f2d40. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:28.496773 (+195845us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:28.496827 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.620616 (+1123789us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54007c1fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:29.620706 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.620794 (+    88us) raft_consensus.cc:1685] finished
0709 12:11:29.620818 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.621237 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43163,"raft.queue_time_us":166239,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":26527,"spinlock_wait_cycles":568176384,"thread_start_us":359891,"threads_started":2,"wal-append.queue_time_us":498042}
I20260709 12:11:29.642473 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:29.644193 14810 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.639655 14559 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:29.650357 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:29.652052 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.652684 14801 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.653591 14787 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.656219 14832 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:29.658653 14640 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.606011 14839 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.670909 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:29.692548 10511 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:29.701772 10583 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:29.641556 14843 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.650292 14838 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.674033 14392 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:29.640166 13946 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.713891 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:29.703416 14832 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:29.674242 10430 rpcz_store.cc:276] 0709 12:11:26.094327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.294705 (+1200378us) service_pool.cc:224] Handling call
0709 12:11:27.295096 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.298535 (+  3439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.298575 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.320405 (+ 21830us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.858261 (+537856us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:27.858306 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.858392 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.521430 (+1663038us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:29.632418 (+110988us) raft_consensus.cc:1685] finished
0709 12:11:29.632457 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.632862 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3261,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":12298,"spinlock_wait_cycles":8161664,"thread_start_us":449161,"threads_started":1,"wal-append.queue_time_us":702471}
I20260709 12:11:29.671433 14706 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.721869 10585 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:29.727727 10584 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:29.728968 14650 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.731247 14787 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.731617 14640 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.733872 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:29.723112 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:29.737633 14840 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.514s	user 0.003s	sys 0.000s
I20260709 12:11:29.739392 14838 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:29.731266 14559 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.742240 14650 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:29.744283 10505 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:29.738698 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:29.744787 14839 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.751642 14868 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:29.753228 10436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 1954) took 1747 ms (client timeout 1659 ms). Trace:
W20260709 12:11:29.753737 10436 rpcz_store.cc:269] 0709 12:11:28.006125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.006274 (+   149us) service_pool.cc:224] Handling call
0709 12:11:29.752776 (+1746502us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b540033c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:29.752987 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558801792}
I20260709 12:11:29.755009 14832 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.755755 10512 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:29.756201 10512 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:29.756419 14832 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.757268 10512 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.718227 14569 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.720729 14801 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:29.760296 14870 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:11:29.712302 14821 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 12:11:29.761276 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 44533.2us
I20260709 12:11:29.739647 14805 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:29.763554 10512 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:29.772328 10586 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:29.779378 14775 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:11:29.781060 10505 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
W20260709 12:11:29.783038 14770 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.007s
I20260709 12:11:29.783694 14770 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.786464 14770 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.752555 14868 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.788358 10505 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:29.789824 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:29.792448 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53765.5us
I20260709 12:11:29.796166 14710 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:29.789873 14868 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:29.801977 10553 connection.cc:256] RPC call timeout handler was delayed by -1.1238284739999926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
I20260709 12:11:29.805383 14720 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:29.816135 14871 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:11:29.816196 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.821321 10506 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:29.820858 14824 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.742846 14559 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:29.831825 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.838395 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:29.814800 14605 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.320s	user 0.005s	sys 0.019s
I20260709 12:11:29.839533 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.717520 14747 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:29.777024 14646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.191s	user 0.012s	sys 0.032s
I20260709 12:11:29.848177 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.844269 14775 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:11:29.849431 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.861481 10504 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:29.865213 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 101647us
W20260709 12:11:29.856077 14646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.357s	user 0.013s	sys 0.032s
I20260709 12:11:29.797035 14710 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.890730 14706 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:29.891135 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:29.892100 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.892935 14824 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.897692 14769 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.877722 10505 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:29.878360 10510 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:29.880149 14239 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.883271 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.884006 14876 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:29.885303 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 89131.6us
I20260709 12:11:29.886408 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:29.888236 14747 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:29.888782 14869 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:11:29.905620 11212 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:29.909711 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:29.915604 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:29.916396 10511 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:29.922063 10586 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:29.900228 14392 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:29.948704 14392 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.949816 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.951416 14787 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:29.955258 14640 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.947626 11217 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:29.959774 14769 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:29.961385 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:29.946899 14839 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:11:29.964071 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:29.948977 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:29.965716 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.964772 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.966842 10506 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
I20260709 12:11:29.966892 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:29.967758 10435 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.967957 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.968077 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:29.969174 10506 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:29.941361 10512 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:29.943810 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:29.971926 10506 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:11:29.971544 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:29.944279 14710 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:29.979166 14868 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:29.979442 14239 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:11:29.942927 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:29.985208 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:29.985155 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 176969us
I20260709 12:11:29.989378 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:29.989684 11216 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:29.992815 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:29.859211 14605 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.001981 10510 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
W20260709 12:11:30.006009 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57875.4us
I20260709 12:11:30.006922 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:30.008615 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:30.011957 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 69277us
I20260709 12:11:30.015461 14805 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:30.019526 14708 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:30.019722 14805 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:30.020141 14708 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:30.021376 14708 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:29.989795 14839 consensus_queue.cc:1243] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:30.016356 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:30.024066 10430 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:30.000888 14747 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.002046 14838 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:30.030886 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:30.029968 11212 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
W20260709 12:11:30.036675 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.039189 14837 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:30.039750 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.040349 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.047384 10507 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:30.047533 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.048509 10552 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:30.052155 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.054896 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:30.039996 14837 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.002766 14704 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.045409 14868 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:30.064283 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:30.071902 14868 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:30.072017 14450 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:30.073426 14450 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:30.073200 14868 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.073913 14450 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:30.075764 14450 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.021329 14805 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.083932 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:30.063822 14839 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:11:30.088346 14704 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.089135 14665 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:30.091873 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:30.111860 10429 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:30.062160 14239 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:30.092772 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:30.093730 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:30.094429 10438 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:11:30.114976 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 50817.5us
I20260709 12:11:30.094918 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:30.101779 10435 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:30.092653 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:30.104249 10436 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:30.089439 14450 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:30.126765 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:30.133543 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:30.103420 14787 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.140215 10430 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:30.067245 14747 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.121120 14710 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:30.118132 14239 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.143015 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:30.094619 14838 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.151328 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:30.153406 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:30.149842 10431 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.160365 10429 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:30.152410 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:30.142894 14845 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.631s	user 0.014s	sys 0.004s
I20260709 12:11:30.174118 10434 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:30.155313 14239 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.170092 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:30.157704 14559 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.183689 10436 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:30.153502 14704 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:30.166170 14710 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:30.200371 10438 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.171207 14845 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.217772 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:30.222069 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57251us
I20260709 12:11:30.191202 14559 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.212381 14605 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.241461 10432 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:30.214525 14837 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.249389 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:30.248939 14640 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:30.255811 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 149406us
W20260709 12:11:30.262342 14885 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:11:30.264132 14746 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:30.264709 14746 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:30.265214 14746 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:30.258481 14838 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.268055 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:30.269387 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:30.258540 14640 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.271248 14640 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.277078 14598 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:30.289909 14821 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:30.291878 14710 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.292512 14833 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.234335 14704 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.290624 14821 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:30.270493 14392 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:30.316171 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:30.317202 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 136647us
W20260709 12:11:30.321316 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51755.8us
I20260709 12:11:30.253988 14877 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 12:11:30.308669 14392 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.340091 14213 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:30.340395 14832 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:30.340772 14213 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:30.341423 14832 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:30.344215 14884 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:11:30.344828 14832 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:30.349239 14883 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.088s	user 0.003s	sys 0.000s
I20260709 12:11:30.350149 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.321844 14769 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:30.357623 14747 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:30.358345 14833 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.359157 14801 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.359807 14845 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.360764 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.363484 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.364431 14704 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.364497 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.365679 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.366477 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.366477 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.367373 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.368068 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.369309 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.413111 14805 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:30.404328 14769 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.417706 14769 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.370806 14515 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:30.370975 14423 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:30.372205 14882 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:11:30.375597 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.342183 14213 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.377631 14842 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:30.380625 10433 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:30.336834 14392 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.388350 14559 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:30.323943 14824 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:30.338510 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:30.402372 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:30.400760 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 49429.9us
I20260709 12:11:30.407593 10587 raft_consensus.cc:2410] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:30.410270 11215 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:30.422254 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 90115.7us
I20260709 12:11:30.433137 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:30.433849 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.446012 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.425171 14515 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:30.470943 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47812.2us
I20260709 12:11:30.436818 14842 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:30.474877 10431 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:30.479668 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.480779 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.447141 14824 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:30.483348 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:30.475835 11212 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:30.483479 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.489312 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.456354 14839 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:30.474028 14515 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:30.491796 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.491917 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.488621 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.494232 10506 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:30.492846 10504 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:30.462800 14559 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 12:11:30.506927 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 168532us
I20260709 12:11:30.510948 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.493892 14787 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:30.513249 10513 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:30.518630 10504 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:30.520246 14904 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.520627 14518 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:30.520474 14839 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.521096 14518 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:30.524799 14824 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.520889 14904 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:30.527081 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.522255 14518 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.533236 14518 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.517880 14901 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:30.517669 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:30.529824 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.536964 11212 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:30.530732 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:30.538335 10513 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 8: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:30.500617 14650 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:11:30.538669 14651 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:30.469583 14239 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.526005 14824 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:30.543232 14842 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.546564 14646 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:30.556600 14646 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:30.557732 14646 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.549382 14423 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:30.559278 14900 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:11:30.559523 14423 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:30.547464 14867 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:11:30.558099 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:30.547438 14559 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:11:30.574152 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46725.1us
I20260709 12:11:30.576200 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.583937 10510 raft_consensus.cc:2410] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:11:30.563499 14839 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.562132 14904 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.576327 14769 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:30.586616 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.589692 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.528787 14877 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:11:30.593027 10507 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:11:30.564563 14559 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:30.604878 14515 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.586222 14569 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.621088 10508 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:30.591171 14646 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.584192 14845 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.622753 14569 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:30.638089 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 120247us
I20260709 12:11:30.640743 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.632596 14518 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:30.655949 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.658861 10507 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:30.634471 14515 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:30.660646 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.628702 14769 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.640012 14569 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.669006 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:30.669709 10511 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:30.642868 14518 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:30.634647 14704 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:30.673535 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:30.673905 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:30.673978 14906 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:11:30.686517 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.680071 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.701036 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.643158 14845 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.702500 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.701895 10513 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:30.708127 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:30.708726 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.709873 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.710892 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.712018 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.712914 10513 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:30.713791 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.714818 10513 raft_consensus.cc:2410] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:11:30.715358 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.711872 14518 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.719107 14914 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:30.719692 14518 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:30.705085 10512 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
W20260709 12:11:30.727810 10438 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.003s
I20260709 12:11:30.728505 10438 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:30.690740 14787 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:30.738198 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.747628 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.748010 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.750535 10584 raft_consensus.cc:2410] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:30.665868 14559 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.732410 14787 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.753975 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.700958 14704 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.764156 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.764905 10580 raft_consensus.cc:2368] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11.
I20260709 12:11:30.767951 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.745689 14877 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:30.768661 10584 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.760990 14787 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.793857 10582 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:30.796363 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:30.797402 14640 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.003s
I20260709 12:11:30.797966 14640 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.781998 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.787513 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.787799 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.788338 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.789546 14704 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.756610 14515 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:11:30.743712 14914 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:30.803689 14913 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:11:30.798494 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.801998 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.802711 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.802788 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:30.803742 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.805114 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:30.806349 14710 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:30.808871 10586 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 12:11:30.842674 14868 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.022s	sys 0.001s
I20260709 12:11:30.842618 14710 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:30.843211 14868 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.022s	sys 0.001s
I20260709 12:11:30.843990 10586 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:11:30.844440 10586 pending_rounds.cc:85] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 4
I20260709 12:11:30.844756 10586 pending_rounds.cc:107] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:11:30.845002 10586 raft_consensus.cc:2889] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:30.845592 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:30.846339 10506 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:30.846374 10478 leader_election.cc:400] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11.
I20260709 12:11:30.850643 10478 leader_election.cc:403] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:30.841842 10580 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:30.778208 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:30.851816 10512 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.814200 10584 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:30.823545 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.822540 11216 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
W20260709 12:11:30.774116 14746 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.508s	user 0.016s	sys 0.000s
I20260709 12:11:30.824712 10587 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:30.810974 14769 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.867339 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.868232 10510 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:30.870005 14239 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.875813 14845 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.861157 14918 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.877015 14918 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.877468 14845 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:11:30.879168 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.880568 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:30.884968 14821 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.888381 14710 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.888288 14845 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:30.857901 14746 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.890185 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:30.891420 14746 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.892071 10587 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:30.894335 14845 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:30.895731 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:30.899818 14845 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.909479 14845 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:30.911669 10581 raft_consensus.cc:2410] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:30.912524 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:30.920084 14845 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:30.921967 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:30.913168 10587 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:30.928633 14559 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.929108 14708 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:11:30.930773 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 74816.7us
I20260709 12:11:30.930912 14821 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.934361 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:30.917129 14640 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:30.935781 10511 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:30.937079 10506 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:30.925354 14914 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.925689 14747 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:30.914228 10580 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:11:30.915233 14908 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:30.917572 14805 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:30.912994 10508 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:30.861948 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:30.943974 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 78245.5us
I20260709 12:11:30.953996 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:30.958190 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.961874 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.959678 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:30.962117 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.963466 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:30.959110 11221 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:30.965301 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.969892 10512 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
W20260709 12:11:30.936638 14775 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.003s	sys 0.033s
W20260709 12:11:30.971490 14775 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.003s	sys 0.033s
I20260709 12:11:30.971323 14868 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11.
I20260709 12:11:30.928627 14845 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:30.974412 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 112586us
I20260709 12:11:30.776301 14877 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:30.985801 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:30.975860 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.974622 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:30.990820 14909 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:11:30.992509 14904 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.992816 14423 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.994087 14569 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.995329 14918 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:30.968904 14392 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:30.997931 14392 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:30.999316 14518 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.999794 14518 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.000185 14518 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.002190 10506 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:31.003602 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:31.004072 10506 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:11:31.004335 11216 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:31.006572 14423 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:30.981271 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 170929us
W20260709 12:11:31.008280 14833 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.001s
I20260709 12:11:30.987059 14832 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.009950 10479 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.011592 14832 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.012391 14833 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:30.960168 14747 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.015285 14747 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.015955 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:31.016289 14747 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:31.016601 14787 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.018368 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.021039 10584 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:31.021092 11216 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:31.023214 10505 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:31.023342 14833 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:31.025700 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:31.026443 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 72660.3us
I20260709 12:11:31.027511 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:31.028301 14795 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:30.983408 14914 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:11:31.034657 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53621.7us
W20260709 12:11:31.039750 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.041036 10508 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:31.041373 13946 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:31.044293 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:31.044483 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:31.045207 11221 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:11:31.018738 14787 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:31.046803 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87249.9us
W20260709 12:11:31.053983 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74310.7us
I20260709 12:11:30.983261 14845 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.082151 14845 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.983974 14708 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:31.088129 14708 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:31.088631 14708 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:31.081023 14566 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:31.089530 14566 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:31.089744 14708 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.062001 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:31.096459 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 70838.5us
W20260709 12:11:31.029834 14810 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.048s	sys 0.045s
I20260709 12:11:31.062600 14769 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:31.053195 14918 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.070209 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:31.100497 11216 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:31.073764 14569 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:31.081167 14877 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:31.091728 14842 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:31.040278 14824 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.061802 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:31.097448 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.033209 14904 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:31.100863 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:31.090674 14566 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:31.099100 14810 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.049s	sys 0.045s
W20260709 12:11:31.107225 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.107278 14708 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.115494 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:31.116246 11212 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:31.117079 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56250.6us
I20260709 12:11:31.094532 14901 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:11:31.100591 14423 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:11:31.123319 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 95648.3us
I20260709 12:11:31.131536 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.132969 14824 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:31.133551 14824 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.134009 14824 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.134243 14845 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.137701 14845 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.139395 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:31.150549 10431 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:11:31.152253 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:31.156594 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.142159 14821 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:31.140982 14423 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.123190 14867 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:31.158913 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.170382 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.161897 14821 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.169195 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:31.140365 14923 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:31.181114 14821 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:31.151713 14868 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:31.189793 10431 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:31.191624 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52414.7us
I20260709 12:11:31.181324 14569 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.192633 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.218344 14518 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.195359 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.195741 10432 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:31.199174 14918 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.236449 14918 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.238018 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.199975 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.238933 14837 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:31.239349 10430 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:31.240602 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.241040 14837 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:31.241315 10430 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:31.243207 14868 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:31.243264 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.181588 14867 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.244454 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.244630 14897 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.245272 14569 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.166971 14924 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:31.247246 14924 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.167860 14775 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:31.206703 14912 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.247646 14569 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:31.188625 14891 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.896s	user 0.005s	sys 0.000s
W20260709 12:11:31.207952 14925 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:11:31.209781 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51002.5us
I20260709 12:11:31.250890 10583 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:11:31.251139 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:31.210536 14423 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:31.220037 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.228713 11212 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:31.193419 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.260118 10436 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 8: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:31.234640 11221 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:31.247787 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.249495 10434 raft_consensus.cc:2368] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12.
I20260709 12:11:31.192515 14910 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.253778 11216 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:31.255053 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 93848.6us
I20260709 12:11:31.255175 10429 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:31.261336 10435 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:31.269840 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:31.271003 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.271307 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.271982 10432 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.272936 14832 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.260712 14838 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:31.273890 14838 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:31.274307 14838 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:31.274906 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.263597 14901 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:31.280591 10436 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:11:31.247643 14924 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.282015 14518 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:31.284075 10434 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:31.285660 10434 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:31.287130 14910 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.250970 14824 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.289749 14910 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.234308 14747 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.291210 14845 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:31.247833 14775 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:31.292690 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.286834 10434 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:31.238816 14918 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.296955 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.291386 14747 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.303077 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.304247 10431 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.307154 10435 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:31.307371 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.309247 14867 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.321354 14213 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:31.323352 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 58646.6us
I20260709 12:11:31.246465 14868 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.335369 14566 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.335597 14775 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.336915 10432 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:31.337172 14775 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:31.339380 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 69684.2us
W20260709 12:11:31.340466 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.340279 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:31.341814 13946 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:31.360605 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.343173 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:31.370167 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62888.2us
I20260709 12:11:31.374120 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.374910 10510 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:31.375378 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.376372 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.376967 10508 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:31.290879 14824 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.377919 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.379341 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.372596 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.295172 14845 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:31.386036 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:31.301225 14901 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.305413 14708 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:31.389216 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46114.7us
I20260709 12:11:31.392139 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.398550 10508 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:31.366055 14650 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:31.403429 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.409058 14650 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.409387 10435 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:31.410421 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.410144 14650 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.397847 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.418601 10505 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:31.403640 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.392738 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:31.420168 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47705.1us
I20260709 12:11:31.420933 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.400363 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.407932 10512 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:31.411083 10435 raft_consensus.cc:2376] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14. Re-sending same reply.
I20260709 12:11:31.411314 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:31.388886 14893 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:31.415710 10584 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:31.416575 10587 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:31.406612 10431 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:31.425496 10509 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:31.407903 11177 leader_election.cc:336] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.618s (SENT)
I20260709 12:11:31.426694 10434 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:31.428027 10584 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 10. Ops: []
I20260709 12:11:31.430528 10584 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:31.430922 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.432883 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:31.429594 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.433810 10583 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:31.404934 14832 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:31.408305 14867 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:31.435042 10584 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:31.435336 10586 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:31.435499 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.436720 10479 leader_election.cc:400] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12.
I20260709 12:11:31.446242 10479 leader_election.cc:403] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:31.432760 14868 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:31.445727 10587 raft_consensus.cc:1217] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 9.7->[11.8-13.9]   Dedup: 11.8->[13.9-13.9]
W20260709 12:11:31.436995 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.433984 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:31.388700 14910 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:11:31.399413 14845 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:31.498029 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50513.2us
I20260709 12:11:31.434520 14821 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.436173 14837 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:31.501205 10587 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:11:31.439662 14747 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:31.507838 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.508602 10438 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:31.509562 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.510504 14821 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:31.441288 14559 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:31.514379 14559 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.515549 14901 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.515942 14559 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.447139 14650 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:31.495509 14868 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.430873 14838 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.525552 14838 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:31.526664 10504 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:31.528503 10505 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:31.500540 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.447598 10435 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:31.502300 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 68415.5us
W20260709 12:11:31.504984 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:31.531095 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.505805 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.531821 10431 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:31.511193 14905 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:31.439828 14907 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:31.512033 10513 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:31.501472 14837 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:31.535051 14867 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:31.535280 14837 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.429551 14824 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:31.528883 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.498868 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:31.532618 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.534833 10507 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:11:31.546948 10435 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:31.539510 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.541059 11217 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:31.540199 14905 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.540064 14867 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.544425 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:31.519554 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:31.549144 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50371.6us
I20260709 12:11:31.433259 14893 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:31.549714 14646 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:31.535676 14907 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:31.442636 14832 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.553260 10438 raft_consensus.cc:2376] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9. Re-sending same reply.
I20260709 12:11:31.491822 10430 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:31.555208 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.568367 10431 raft_consensus.cc:2410] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:31.555632 10587 raft_consensus.cc:2410] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:11:31.560454 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:31.561345 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 116249us
I20260709 12:11:31.549288 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.569626 10435 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:31.577041 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.577982 10430 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11. Re-sending same reply.
I20260709 12:11:31.574420 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.586109 10436 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:31.588124 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:31.588908 14213 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.592113 14930 raft_consensus.cc:2764] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:11:31.598820 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:31.443682 14924 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.604946 10433 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:11:31.606209 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:31.630352 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.603564 10432 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:31.611526 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:31.633611 14213 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:31.635871 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.636675 10436 raft_consensus.cc:2376] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14. Re-sending same reply.
I20260709 12:11:31.588949 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.564759 14907 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:31.573572 14832 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.612737 14824 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:31.654243 14213 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:31.606936 14646 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:31.661396 14213 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:31.659626 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 256791us
I20260709 12:11:31.631371 14914 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:31.665004 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 76140us
I20260709 12:11:31.665799 10583 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:31.665956 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.666221 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:31.666487 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.669219 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:31.632016 14821 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:11:31.657173 14842 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:31.673902 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.678504 14239 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.680464 10433 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:31.612178 14905 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.702004 14213 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.703086 14932 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:11:31.724411 14924 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:31.706740 14893 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.710014 14423 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:31.677052 14515 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:31.764298 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.695523 14824 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:31.767894 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 2080) took 1069 ms. Trace:
I20260709 12:11:31.776633 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:31.769760 14213 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:31.770421 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 64694.7us
I20260709 12:11:31.772423 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:31.774050 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:31.675143 14914 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.780050 10509 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:31.771678 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:31.778474 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.774940 10507 rpcz_store.cc:276] 0709 12:11:30.684631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.690296 (+  5665us) service_pool.cc:224] Handling call
0709 12:11:31.753357 (+1063061us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b5400412654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:31.753563 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899095424}
I20260709 12:11:31.827867 10581 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:31.828675 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 50309.7us
I20260709 12:11:31.838294 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.839339 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.840301 10429 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:31.840476 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.841347 10436 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:31.846060 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.846894 10431 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:31.847205 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:31.848084 10433 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:31.773573 14824 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:31.831293 14810 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.885911 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:31.878305 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.886946 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:31.874734 14842 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:31.895136 10437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.031s
I20260709 12:11:31.846377 14515 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:31.899978 10437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.007s	sys 0.032s
I20260709 12:11:31.848546 14559 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.901716 10436 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:31.899497 14515 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:31.903967 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:31.904664 14566 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:31.906522 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 121089us
I20260709 12:11:31.874969 14867 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.909371 14832 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:31.916044 14824 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.917173 14810 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:31.929608 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:31.941672 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:31.946743 10510 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.000s
I20260709 12:11:31.947392 10510 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:31.948586 10429 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:31.864310 14450 consensus_queue.cc:1059] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 12:11:31.952381 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 3.228s (SENT) [suppressed 38 similar messages]
I20260709 12:11:31.988417 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:31.989835 10436 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:31.936708 14775 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:31.996237 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 92376us
I20260709 12:11:32.013162 14810 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:31.987809 14878 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.997852 10429 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:32.029022 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.030122 10429 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:32.030897 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.932610 14559 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.035668 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:31.937543 14867 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:31.993901 14824 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:31.925990 14646 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.014606 14878 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.049420 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.048380 14775 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:32.046648 11216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.015s	sys 0.022s
W20260709 12:11:32.052919 11216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.015s	sys 0.023s
I20260709 12:11:32.072090 14878 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.073866 14832 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.083580 14559 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.085054 14914 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:32.046260 14515 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.086972 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:32.088166 11221 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:32.090898 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.091141 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.091907 10436 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:32.094058 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.095827 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.096616 10431 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:32.118088 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89614.8us
I20260709 12:11:32.121999 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.138123 14907 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.132105 14566 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:11:32.141633 14867 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:32.133435 14868 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.004s
I20260709 12:11:32.146436 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.147508 10432 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
W20260709 12:11:32.150286 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:32.143671 14948 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.092s	user 0.001s	sys 0.003s
I20260709 12:11:32.159157 14868 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12.
W20260709 12:11:32.168485 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 227329us
W20260709 12:11:32.167048 14837 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.009s
W20260709 12:11:32.179610 10553 leader_election.cc:336] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.507s (SENT)
I20260709 12:11:32.180778 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.181568 10581 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:32.172439 14515 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.191419 14515 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:32.194468 14650 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.675s	user 0.014s	sys 0.000s
I20260709 12:11:32.195497 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:32.198753 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.172987 14907 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:32.205427 14867 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.206054 14867 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.209097 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.210014 10429 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:32.213809 10553 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.515s (SENT)
I20260709 12:11:32.174145 14910 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.219444 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.220196 10507 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:32.223670 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.224598 10510 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:32.219492 14910 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.224953 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.225646 10512 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:32.225732 14867 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:32.227341 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.227766 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.230561 10436 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
W20260709 12:11:32.231316 10553 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.659s (SENT)
I20260709 12:11:32.232175 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.239939 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.240691 10430 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
W20260709 12:11:32.240950 10553 leader_election.cc:336] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.825s (SENT)
I20260709 12:11:32.243214 11217 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:32.226785 14910 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:32.243747 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:32.226786 14944 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:11:32.183413 14837 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.252833 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.253631 10437 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 3.
I20260709 12:11:32.253494 10429 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:32.260608 14805 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:32.260914 14837 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:32.261104 14805 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:32.261727 10513 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:32.267134 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.277416 10586 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:32.243559 14867 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:32.292433 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.293295 10437 raft_consensus.cc:2376] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9. Re-sending same reply.
W20260709 12:11:32.299134 14947 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.159s	user 0.002s	sys 0.002s
I20260709 12:11:32.256246 14959 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:32.305727 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.307565 14867 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.311138 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.311689 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.312609 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:32.312811 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 99043.6us
I20260709 12:11:32.313354 10431 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9.
I20260709 12:11:32.313983 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.314579 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.315326 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.314580 10435 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.319765 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:32.321388 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.321447 14905 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:32.322597 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.363498 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.323607 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.365484 10504 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.323037 10434 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:32.333482 10435 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:32.328166 10510 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:11:32.323201 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.370626 10437 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:11:32.323316 10507 raft_consensus.cc:2376] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11. Re-sending same reply.
I20260709 12:11:32.376087 10552 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:32.324369 14941 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.478s	user 0.002s	sys 0.000s
I20260709 12:11:32.318595 14959 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:32.381866 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.383116 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:32.383795 10553 leader_election.cc:336] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.684s (SENT)
I20260709 12:11:32.353299 14559 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.388599 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.362972 14907 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.388016 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:32.262141 14805 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.353322 14905 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:32.392499 14213 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:11:32.449119 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61188.2us
I20260709 12:11:32.450405 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.483857 14213 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:32.484877 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:32.387634 14904 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:32.506647 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.507473 10435 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:32.392285 14912 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:32.495011 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.495730 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.514318 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.481376 14450 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:32.390945 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:32.525920 10438 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:32.526844 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.388747 14559 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.528091 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.529300 10436 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:32.487978 14824 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:32.527859 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.540125 10512 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:32.534431 10434 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:32.535616 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 144742us
I20260709 12:11:32.536973 10511 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:11:32.538089 10510 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:32.538275 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.543166 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.543808 10508 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:32.502923 14910 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:32.504741 14904 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.544412 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.585618 10431 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:32.585704 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.593654 10507 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:32.511602 14878 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:32.594620 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52086us
I20260709 12:11:32.481446 14907 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:32.599714 14963 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.246s	user 0.001s	sys 0.001s
I20260709 12:11:32.600395 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 755) took 1206 ms. Trace:
I20260709 12:11:32.600782 10578 rpcz_store.cc:276] 0709 12:11:31.394179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.394309 (+   130us) service_pool.cc:224] Handling call
0709 12:11:31.397830 (+  3521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:31.433660 (+ 35830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:31.433697 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:31.439118 (+  5421us) log.cc:844] Serialized 26 byte log entry
0709 12:11:31.648048 (+208930us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:31.648100 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.648183 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:32.323616 (+675433us) spinlock_profiling.cc:243] Waited 8.86 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:32.589552 (+265936us) raft_consensus.cc:1685] finished
0709 12:11:32.589576 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:32.600320 (+ 10744us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19863,"raft.queue_time_us":28541,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":20379392,"thread_start_us":208449,"threads_started":1,"wal-append.queue_time_us":259529}
I20260709 12:11:32.595769 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:32.530316 14450 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:32.604265 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.605368 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:32.608772 14213 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:32.610050 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.614611 14213 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.615079 14213 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:32.618476 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.619313 10584 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:11:32.619578 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.620491 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.623937 10585 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:32.543509 14901 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:32.543661 14824 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:32.625983 10504 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:32.625880 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:32.633025 10508 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:11:32.638613 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.688s (SENT)
I20260709 12:11:32.638319 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:32.639982 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.640698 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.641873 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.528607 14930 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:11:32.642757 10586 raft_consensus.cc:2410] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:32.595326 14904 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:32.643733 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.649194 10587 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:32.649670 10580 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:32.639189 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:32.647433 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:32.652281 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 123846us
I20260709 12:11:32.620023 14450 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:32.612773 14910 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:32.641440 14824 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:32.660022 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.636899 14901 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.639178 14969 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:32.663859 10553 leader_election.cc:336] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.731s (SENT)
I20260709 12:11:32.648087 10384 placement_policy_util.cc:248] tablet 0b474d9b241d4a698702287ee1bbe3b2: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 12:11:32.671985 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.670594 14213 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.679644 10507 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:32.681313 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.687393 14969 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:32.689561 11178 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.880s (SENT)
I20260709 12:11:32.690475 14969 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.691246 14912 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:32.692008 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.692662 10507 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:32.693559 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:32.695678 10553 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.748s (SENT)
I20260709 12:11:32.696174 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.696679 14842 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:32.697252 14842 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.697726 14842 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:32.697646 14969 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:32.698961 14515 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.700901 14665 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:32.701330 14665 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:32.702222 14665 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.704592 14960 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:32.707623 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2113) took 1076 ms. Trace:
I20260709 12:11:32.708187 10513 rpcz_store.cc:276] 0709 12:11:31.631033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.631241 (+   208us) service_pool.cc:224] Handling call
0709 12:11:31.631745 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:32.262166 (+630421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.707191 (+445025us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b5400412b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:32.707478 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448286592}
I20260709 12:11:32.709509 14213 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:32.706804 14805 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:32.712186 14960 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:11:32.713430 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:32.714207 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.716339 14910 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.717729 14877 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.718298 14877 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.720616 14877 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.723013 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.724987 14907 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:32.725272 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.725477 10578 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:11:32.725927 10578 pending_rounds.cc:85] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:11:32.726243 10578 pending_rounds.cc:107] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:11:32.726512 10578 raft_consensus.cc:2889] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:32.727276 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.727998 10578 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:32.728524 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.729000 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.729717 10578 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:32.729663 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.725997 10584 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:32.731065 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.731721 10582 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:32.731935 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.732657 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.733091 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.733323 10582 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:32.734117 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.667793 14924 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:32.739343 14867 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:32.671731 14901 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:32.741217 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.742285 10512 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:32.620728 14878 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.744208 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.745251 10509 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:32.745608 14878 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:32.655283 14905 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.737905 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.734458 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.747826 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.748538 10582 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:32.748844 10508 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:32.749063 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.749307 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.750217 10508 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:32.750407 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.751034 10582 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:32.751763 10583 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:32.751886 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.752736 14969 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:32.730786 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.753130 14969 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:32.753163 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.763692 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.767333 10582 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:11:32.771615 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47475us
I20260709 12:11:32.774648 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:32.772846 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:32.776790 11221 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:11:32.788645 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.792488 14845 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:32.750237 14924 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:32.771481 14566 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:32.755203 14969 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.779204 14960 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:32.773831 14878 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:11:32.796469 14930 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:32.815640 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 189840us
I20260709 12:11:32.780200 14423 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:11:32.810964 14747 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:32.869858 10403 connection.cc:256] RPC call timeout handler was delayed by -1.3565093240001715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 12:11:32.870561 10403 connection.cc:256] RPC call timeout handler was delayed by -1.3259093240001221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:11:32.870961 10403 connection.cc:256] RPC call timeout handler was delayed by -1.3160093240001061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260709 12:11:32.871383 10403 connection.cc:256] RPC call timeout handler was delayed by -1.2260093240001879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16570000000000001s for a response
W20260709 12:11:32.871762 10403 connection.cc:256] RPC call timeout handler was delayed by -1.184609324000121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 12:11:32.872157 10403 connection.cc:256] RPC call timeout handler was delayed by -1.1117093240000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
I20260709 12:11:32.799731 14959 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:11:32.836522 14912 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:32.889961 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:32.895730 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 133493us
I20260709 12:11:32.805972 14845 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.897513 14969 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.903417 14979 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:32.908808 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:32.874434 14918 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.912916 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.913755 10429 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:32.914045 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:32.914893 10432 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:32.926008 10582 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:11:32.926579 10582 pending_rounds.cc:85] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 5
I20260709 12:11:32.926496 14213 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:32.927202 10582 pending_rounds.cc:107] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:11:32.927618 10582 raft_consensus.cc:2889] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:32.929787 14213 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:32.885677 14559 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:32.936548 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:32.940527 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.893702 14912 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.862717 14867 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:11:32.948339 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.949496 10513 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.950273 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.941632 10582 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:32.845687 14930 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.960873 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.955402 10504 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:32.924563 14907 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:32.951601 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:32.969333 10437 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:32.969491 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:32.970283 10504 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:32.970188 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:32.883527 14747 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:32.956110 10507 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
W20260709 12:11:32.970196 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 255982us
I20260709 12:11:32.971500 10581 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:32.966708 10510 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:32.967460 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:32.955919 14951 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:11:32.953788 14982 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:32.907421 14810 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:11:32.952977 14912 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:32.978331 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:32.977159 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 68451.9us
I20260709 12:11:32.985426 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:32.954262 14893 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:32.979904 14982 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:32.958117 14972 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:32.988169 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:32.987747 14893 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:32.935411 14918 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:32.993659 14893 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:33.006603 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 92956us
I20260709 12:11:32.990134 10432 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:32.991677 14960 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:32.991183 14837 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:33.052380 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50893.5us
I20260709 12:11:33.053610 10436 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:11:33.064895 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57155.5us
I20260709 12:11:33.068080 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.070061 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:33.008723 14646 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:33.078228 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 92946.2us
I20260709 12:11:33.055141 14960 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.079173 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.080720 10511 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
W20260709 12:11:33.082965 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 310200us
I20260709 12:11:33.084375 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.066355 14867 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:11:33.085639 14845 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.084741 14837 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:33.087870 14837 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:33.088228 14837 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:33.003837 14982 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:33.091637 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.091823 14982 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.084004 14960 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:33.095623 14973 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:11:32.999657 14971 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
W20260709 12:11:33.099514 14970 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:11:33.102748 14950 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.971s	user 0.000s	sys 0.017s
I20260709 12:11:33.104123 14837 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.109894 14930 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.111995 10513 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:33.113622 14837 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.117269 10583 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:33.117733 14960 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.069041 14569 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:33.128978 14918 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.135197 14646 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:33.069813 14907 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:11:33.138368 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.139894 10587 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
W20260709 12:11:33.159685 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75407us
W20260709 12:11:33.142836 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 63763.7us
I20260709 12:11:33.147074 14845 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.110267 14904 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.164539 10578 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:33.165241 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.172067 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.195801 10584 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:33.152351 14930 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.198132 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.150905 14646 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.162367 14905 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.202528 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.195804 14805 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.207860 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.197146 14983 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.159428 14984 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:33.209559 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 139658us
I20260709 12:11:33.212841 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.227510 10552 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.202597 14918 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:33.218683 14213 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.233458 14213 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.231070 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.229208 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:33.237149 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 72067.9us
I20260709 12:11:33.235339 11215 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:33.205307 14646 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:33.243263 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.205366 14878 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:33.205107 14985 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.202042 14986 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:33.254496 14213 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.254787 10430 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:33.256800 11221 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:33.234862 14805 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.232259 14824 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.247255 14867 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:33.253899 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:33.263582 10513 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:33.264004 14907 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.263584 14867 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.264595 14907 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.266753 14867 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:33.267283 10384 auto_rebalancer.cc:629] Failed to schedule move for tablet b117a8eaf2494d0ea2317e5895cac532: Illegal state: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:33.270300 14824 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:33.270884 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.271982 14907 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.273839 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.222745 14924 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:33.264268 10434 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:33.274899 11215 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:33.243665 14515 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:33.285476 14515 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.285943 14515 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.242683 14837 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:33.287536 14837 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:33.288642 14837 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:33.290768 14842 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.002s
I20260709 12:11:33.205406 14930 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.242976 14971 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:33.293296 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.243553 14978 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:33.245465 14987 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.299007 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:33.284726 14901 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.000s
I20260709 12:11:33.314589 14867 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:33.319840 14867 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.327582 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.274390 14905 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:11:33.313000 14978 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.334103 14978 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.274659 14985 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.328742 14904 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.345886 14985 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.300649 14842 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.322386 14987 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.239331 14984 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.312716 14910 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:33.354027 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.351878 10512 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:33.331027 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 263017us
I20260709 12:11:33.332851 14805 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.333420 14960 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:33.376543 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49662.3us
W20260709 12:11:33.376770 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 148080us
I20260709 12:11:33.350768 14842 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.377894 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.369261 14845 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:33.368211 14910 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.369057 14918 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:33.332505 14983 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:33.376576 14991 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 12:11:33.314157 14971 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.276916 14986 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.366838 14824 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:33.408950 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 165780us
I20260709 12:11:33.324892 14901 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.366671 14975 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:33.412101 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46193.8us
I20260709 12:11:33.432260 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.408391 14910 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:33.410849 14994 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:33.404925 14747 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:33.426287 14450 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.006s
I20260709 12:11:33.443717 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.444480 10506 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:33.416203 14810 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:11:33.459030 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.459704 10506 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:33.459527 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.419607 14842 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:11:33.413528 14845 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.461020 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.421624 14983 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.444199 14975 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:33.469982 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 60214.8us
I20260709 12:11:33.453979 14985 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:33.458279 14989 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 12:11:33.488482 14985 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.491600 14912 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.493191 10586 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:33.495100 10505 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:33.444399 14997 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:33.488642 14987 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.511354 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:33.512089 10510 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:33.435026 14824 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.520247 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.521461 14824 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.477684 14986 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.442389 14450 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.523653 14971 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.525632 14450 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.479516 14930 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.526672 10585 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:33.479761 14901 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.460583 14969 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:33.529278 10584 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:33.455857 14877 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:11:33.484826 14845 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.485828 14984 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:33.470779 14992 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:33.489801 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.538673 10510 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:33.542114 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.542248 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.543182 11215 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:33.488840 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.498133 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.488505 15005 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:33.494520 14985 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.500032 14975 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.486927 14978 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.494462 14912 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.514137 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:33.514765 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:33.561630 10504 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:33.515509 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:33.474048 14867 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.534782 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:33.564541 10506 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:33.547297 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.546847 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.553891 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.555563 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.557181 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:33.581809 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 93074us
I20260709 12:11:33.588277 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.588428 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.571480 10580 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:33.589493 10505 raft_consensus.cc:1240] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:33.575839 10504 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:33.539790 14997 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:33.557088 15005 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.586355 10510 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:33.571614 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:33.597606 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:33.598356 10581 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:33.590718 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:33.600445 10506 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:33.557561 14845 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:33.598966 10578 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:33.589404 10582 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:33.597678 10552 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.592685 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.605186 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.539496 14959 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:11:33.607481 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.608417 10582 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:33.609608 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.609620 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.610873 10582 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:33.613047 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.613476 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.614120 10580 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:33.614957 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.605360 10579 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:33.612228 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.612762 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:33.626873 10583 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:33.621281 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.596433 15005 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:33.635188 14997 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.637930 14983 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.561080 14992 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.618038 14914 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.611373 14986 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:33.642643 10506 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:33.621074 14979 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:33.629554 14968 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.620834 14984 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:33.620652 14930 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:33.647056 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.652187 11216 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
W20260709 12:11:33.654102 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.653991 11215 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:33.645560 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.644289 15001 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:33.649653 14984 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:33.659106 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.650591 14877 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:33.651871 14930 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.655836 14979 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:33.658974 15001 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.661017 14984 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.684263 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.694743 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.646727 14914 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.698534 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:33.698839 14978 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.694444 15001 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:33.683008 14930 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:33.692130 14968 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:33.702307 10506 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:33.696743 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.701406 10509 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:11:33.706823 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:33.710186 10510 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.703322 14842 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:33.712548 14842 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.712982 14842 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:33.718497 14914 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.726917 14907 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:33.727497 14907 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.727850 14907 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:33.728749 15001 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.729553 14910 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.735044 14665 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.697057 14974 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:33.691883 14877 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:33.738451 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:33.748925 11178 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.810s (SENT)
I20260709 12:11:33.764837 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:33.737623 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.762794 10581 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:33.692396 14985 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:11:33.768412 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:33.763875 14970 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:11:33.761528 14974 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.768455 14987 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:33.770649 10510 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
W20260709 12:11:33.792948 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44103.9us
I20260709 12:11:32.601459  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:33.773849 14867 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:33.771677 14968 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:33.779714 14868 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:33.800668 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.783145 14978 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:11:33.810899 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72668.7us
W20260709 12:11:33.811328 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 192277us
I20260709 12:11:33.791462 14974 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:33.788048 14893 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.788261 15001 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.799854 14867 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.836477 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:33.784008 15009 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:11:33.841985 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68579.3us
I20260709 12:11:33.827239 14978 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.829473 14912 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:33.852303 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:33.857044 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:33.836104 14867 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.855564 14910 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:33.842833 14868 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:33.863273 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:33.897327 14569 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:33.919251 14569 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:33.919821 14515 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.005s
I20260709 12:11:33.920215 14912 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:33.920516 14569 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.921494 14912 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.852494 14988 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:11:33.923874 14988 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:33.924360 14988 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:33.850749 14914 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:33.899238 14910 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:33.901611 15005 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.903440 14930 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.854193 14986 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 12:11:33.904752 14646 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.002s
I20260709 12:11:33.776654 14979 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.908643 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:33.853657 14978 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:33.831858 14877 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.913623 14968 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.897850 14993 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:33.853765 14992 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:33.929567 14988 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.935927 10510 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:11:33.943770 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67526.4us
I20260709 12:11:33.906445 14842 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.968207 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:33.930137 14867 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.947782 14914 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.979193 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:33.975725 14972 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:33.920369 14515 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:33.938680 14907 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:33.991689 11212 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:11:33.974476 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 65922.3us
I20260709 12:11:33.978854 11217 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:34.003403 10403 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:34.005848 15018 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:11:33.968593 14910 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:34.035953 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.996925 14914 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.998106 14972 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.977067 14646 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:34.050932 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44276.7us
I20260709 12:11:34.001183 14987 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.057642 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:33.996594 14904 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:11:34.020850 14988 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:34.075197 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.076546 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.076823 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.077828 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.078140 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.078755 10432 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:34.079751 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.080546 10431 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:34.077884 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.081362 10437 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:34.078549 10434 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:34.051275 14972 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:33.996258 15007 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:11:34.095021 10334 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:34.097074 10334 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 2 to 10. New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.103506 14842 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:34.107659 15027 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:11:34.108863 14907 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:34.109257 15024 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:11:34.111222 15005 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:34.112779 10331 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 109) took 16008 ms (client timeout 15000 ms). Trace:
I20260709 12:11:34.113497 14910 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:34.067821 14984 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:34.115967 14867 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.114705 14646 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.119159 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.098661 10334 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 6 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } }
I20260709 12:11:34.123951 10431 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:34.007249 14868 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.131883 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:34.132566 10586 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:34.139081 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.117981 14515 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:34.138338 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 143760us
I20260709 12:11:34.118470 14986 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.118122 14984 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.150023 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:34.160625 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:34.169134 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.834s (SENT)
I20260709 12:11:34.169804 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.170706 14515 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.145107 14959 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:11:34.171154 10553 leader_election.cc:336] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.854s (SENT)
I20260709 12:11:34.171350 14959 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:34.171705 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.174069 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.177201 14566 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:34.178128 14566 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:34.180380 10508 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:34.185245 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.185355 10508 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:34.188921 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.189306 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.190169 10513 raft_consensus.cc:2393] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:34.191859 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.194888 10430 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:34.196816 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.197472 10583 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:34.197940 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.158185 10334 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 2 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.201465 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.203682 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.138653 14994 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.208841 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.209960 10506 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:34.206135 10510 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:34.206845 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.212078 10430 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:34.219521 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.219528 10334 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 8, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.204447 10508 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:34.207491 10586 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:34.233862 10430 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:34.235268 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.239490 10430 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:34.241288 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.243355 10430 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:34.257154 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.265051 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.265890 10431 raft_consensus.cc:2376] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14. Re-sending same reply.
I20260709 12:11:34.216483 14559 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.268980 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.226511 14515 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.272073 10430 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:34.273430 10431 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:34.287612 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.213577 14994 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:34.299916 10334 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.267567 14569 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:34.353420 10430 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:34.356442 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.357226 10430 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:34.359962 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:34.363430 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 105794us
I20260709 12:11:34.364349 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:34.363152 14559 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.366595 14559 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.368216 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.373782 14559 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:34.374315 15033 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:11:34.358649 15019 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:11:34.374974 10334 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 6 to 8, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.305413 14974 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.306886 14986 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:11:34.402433 10334 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.409962 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.365444 14569 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.396617 14993 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:34.416744 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48612.2us
I20260709 12:11:34.413246 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.414373 10334 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 8, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.419159 10430 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:34.420115 10403 leader_election.cc:400] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:34.420743 10403 leader_election.cc:403] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:11:34.425321 10334 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.432889 10334 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.435179 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.441319 10334 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 6 to 7, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.456183 10334 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 3 to 6, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 6 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.431275 14930 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.471618 10431 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:34.485081 10334 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:11:34.476667 15036 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:11:34.488178 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:34.478341 14986 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.481218 15037 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.520728 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:34.515170 10334 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.516446 15051 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:34.527119 15051 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:34.528842 14569 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.530033 15051 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:34.516458 15056 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:34.521824 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:34.531133 10510 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:34.521588 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:34.524096 14910 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:34.523442 10334 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 3 to 6, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 6 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:34.474385 15055 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:34.483088 15053 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:34.489210 14646 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.525907 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.529760 15037 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.523356 14837 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:34.536471 10513 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:34.525763 15041 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:11:34.514865 14997 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:34.574766 10334 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 8, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.574028 10510 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:34.593878 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2611) took 1062 ms. Trace:
W20260709 12:11:34.595407 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 177631us
I20260709 12:11:34.598510 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:34.599941 10513 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:34.558524 15057 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.595033 10585 rpcz_store.cc:276] 0709 12:11:33.531214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.532781 (+  1567us) service_pool.cc:224] Handling call
0709 12:11:33.533128 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:33.533531 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:33.559789 (+ 26258us) log.cc:844] Serialized 14 byte log entry
0709 12:11:33.681555 (+121766us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:34.421948 (+740393us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:34.422432 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:11:34.422607 (+   175us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:34.422644 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:34.422723 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:34.587810 (+165087us) raft_consensus.cc:1685] finished
0709 12:11:34.592198 (+  4388us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:34.593776 (+  1578us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26231,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":2080,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":21445,"spinlock_wait_cycles":35642112,"thread_start_us":79216,"threads_started":1,"wal-append.queue_time_us":107135}
I20260709 12:11:34.477036 14993 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.560811 14994 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:34.575675 15040 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:11:34.596956 14997 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:34.600960 14974 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.616541 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.607589 10334 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 6 to 3, term changed from 8 to 7, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) added. New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:34.564333 14646 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:34.620009 14997 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:34.615393 14994 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:34.648386 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.649093 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.564440 15053 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.656116 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2622) took 1072 ms. Trace:
I20260709 12:11:34.656538 10587 rpcz_store.cc:276] 0709 12:11:33.583359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.597832 (+ 14473us) service_pool.cc:224] Handling call
0709 12:11:33.618540 (+ 20708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:33.618930 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:33.618965 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:33.923903 (+304938us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b18001759c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a
0709 12:11:33.924598 (+   695us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:33.925139 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:11:34.208028 (+282889us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:34.208076 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:34.229617 (+ 21541us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:34.655859 (+426242us) raft_consensus.cc:1685] finished
0709 12:11:34.655887 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:34.656043 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19236,"raft.queue_time_us":1307,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2871,"spinlock_wait_cycles":21979776,"thread_start_us":217110,"threads_started":1,"wal-append.queue_time_us":295003}
I20260709 12:11:34.641417 15055 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:34.609534 15057 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:34.669857 15052 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:11:34.668845 15055 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:34.664294 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:34.651659 15059 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.666832 10334 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.673974 15059 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:34.675412 14972 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.002s
I20260709 12:11:34.676317 15059 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.679874 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:34.113202 10331 rpcz_store.cc:269] 0709 12:11:18.104375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.688062 (+583687us) service_pool.cc:224] Handling call
0709 12:11:34.112522 (+15424460us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:28.288028 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:28.288946 (+   918us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:28.289001 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:11:28.289029 (+    28us) tablet.cc:662] Decoding operations
0709 12:11:28.322021 (+ 32992us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:28.335849 (+ 13828us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:28.339343 (+  3494us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:11:28.343960 (+  4617us) tablet.cc:701] Row locks acquired
0709 12:11:28.343992 (+    32us) write_op.cc:260] PREPARE: finished
0709 12:11:28.344235 (+   243us) write_op.cc:270] Start()
0709 12:11:28.344370 (+   135us) write_op.cc:276] Timestamp: P: 1783599088344148 usec, L: 0
0709 12:11:28.344401 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:11:28.345410 (+  1009us) log.cc:844] Serialized 5958 byte log entry
0709 12:11:29.260792 (+915382us) op_driver.cc:464] REPLICATION: finished
0709 12:11:29.275042 (+ 14250us) write_op.cc:301] APPLY: starting
0709 12:11:29.275151 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:29.296781 (+ 21630us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:29.296874 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:29.487543 (+190669us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:29.589214 (+101671us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:11:29.630753 (+ 41539us) write_op.cc:312] APPLY: finished
0709 12:11:29.631774 (+  1021us) log.cc:844] Serialized 139 byte log entry
0709 12:11:29.632143 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:29.632773 (+   630us) write_op.cc:454] Released schema lock
0709 12:11:29.632878 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:29.673473 (+ 40595us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:29.674549 (+  1076us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":282,"mutex_wait_us":1107090,"reported_tablets":20,"spinlock_wait_cycles":71424,"child_traces":[["op",{"apply.queue_time_us":13879,"apply.run_cpu_time_us":23231,"apply.run_wall_time_us":358562,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66857,"num_ops":15,"prepare.queue_time_us":320780,"prepare.run_cpu_time_us":49463,"prepare.run_wall_time_us":579484,"raft.queue_time_us":186683,"raft.run_cpu_time_us":1439,"raft.run_wall_time_us":20412,"replication_time_us":916166,"thread_start_us":822015,"threads_started":4,"wal-append.queue_time_us":661962,"wal-append.run_cpu_time_us":30805,"wal-append.run_wall_time_us":1618855}]]}
I20260709 12:11:34.681828 10431 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:34.682785 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:34.684271 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.664489 15061 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.682052 10334 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:34.690253 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.670560 15055 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:34.676345 15050 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:11:34.693620 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:34.685679 15046 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:11:34.696709 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.696980 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.686475 14994 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:34.699883 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.700503 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.701105 10430 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:34.701031 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.698496 14845 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.703820 15053 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.704528 15037 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.706329 15059 raft_consensus.cc:2764] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:34.707798 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.706847 15057 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:34.711961 14646 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:34.713989 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:34.700161 14988 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.614s	user 0.015s	sys 0.000s
I20260709 12:11:34.715396 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.683162 14972 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.707697 15061 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.719568 10436 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:11:34.708457 14845 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.721752 10436 pending_rounds.cc:85] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:11:34.724954 10436 pending_rounds.cc:107] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:11:34.713789 15059 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.725301 10436 raft_consensus.cc:2889] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:34.716759 14646 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.727757 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.716364 15037 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.730370 10430 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:34.718958 14980 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.745065 14901 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:34.745183 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.747670 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.721118 15067 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.748217 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.750311 14665 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:34.750753 14665 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:34.748960 14905 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.724804 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:34.752506 14665 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.753506 10513 raft_consensus.cc:606] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.753724 14665 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:34.753921 10513 raft_consensus.cc:614] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:34.755381 14972 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:34.761802 15049 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:11:34.747996 14980 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.764557 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.770145 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.769882 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.783982 14901 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.725919 14994 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:11:34.769497 14930 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:34.785986 14905 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:34.789038 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:34.789245 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.789665 10513 raft_consensus.cc:606] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.727308 14988 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:11:34.789868 10431 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:34.790030 10513 raft_consensus.cc:614] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:34.744056 15057 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:34.790934 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.792800 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.793824 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.786108 14930 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:34.795243 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.796811 14845 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:11:34.769337 14972 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:11:34.804955 14959 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:34.806617 14980 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:34.783229 14515 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:34.799916 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.815846 10431 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:34.816696 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.817274 10431 raft_consensus.cc:2376] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 9. Re-sending same reply.
W20260709 12:11:34.817934 14978 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.834s	user 0.014s	sys 0.000s
I20260709 12:11:34.818147 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.818450 14978 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:34.818831 10431 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:34.821424 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.821938 10431 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:34.822391 14845 consensus_queue.cc:1243] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:34.802767 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:34.824149 10513 raft_consensus.cc:606] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:34.824565 10513 raft_consensus.cc:614] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:34.811076 14988 consensus_queue.cc:1243] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:34.825564 15061 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:34.745929 15037 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.808287 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.807075 14959 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:34.743415 14646 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.750761 15059 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:11:34.815001 14515 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:34.829149 10430 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:34.864128 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.865024 10430 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:34.872625 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.880024 15037 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:34.881338 15069 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:11:34.881755 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.881760 14930 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.866315 14515 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:34.883364 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:34.885257 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 89234.4us
I20260709 12:11:34.886336 10435 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:34.890223 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:34.891490 14901 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:34.892097 14901 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.892597 14901 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:34.893656 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:34.894968 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.895257 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.897027 10435 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:34.897838 10431 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:34.899072 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.899940 10430 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:34.911939 15060 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.916950 15060 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:34.920418 15068 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:11:34.923579 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.925580 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.926740 10583 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:34.927858 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.928437 10587 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:34.929811 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.932950 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:34.933450 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:34.934094 10509 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:34.934644 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.935628 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.936489 15055 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:34.937717 10433 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
W20260709 12:11:34.897440 14914 consensus_meta.cc:287] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.834s	user 0.015s	sys 0.000s
I20260709 12:11:34.940536 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.946288 10504 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:34.940845 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:34.941562 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.947448 10586 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:34.941694 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:34.948405 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.942538 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.949750 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:34.913678 14978 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:34.952519 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.933576 10434 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:34.952978 10512 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:34.952790 10323 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e4cd5d330f694dacb199bd61266c8f5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 52bae2d2caf94d97b1049ecd4f82a2df is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:34.954792 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.955734 10433 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:34.956974 10585 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:11:34.951117 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:34.952662 15001 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:11:34.957412 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:34.966140 10581 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:34.945482 10432 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:34.958586 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:34.962368 10509 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:34.971822 10430 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:34.963596 10435 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:34.950790 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:35.057371 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 106641us
I20260709 12:11:35.058894 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:34.972008 10586 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:34.972632 14918 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:34.979768 15067 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:11:34.981140 15042 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.596s	user 0.001s	sys 0.001s
I20260709 12:11:34.982370 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:35.073508 11216 raft_consensus.cc:2410] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:11:34.964965 14994 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.983218 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:34.986153 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:34.938838 15055 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.988245 14646 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.956883 15057 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.989466 15060 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:34.989476 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:35.083078 10433 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:34.990914 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:35.084349 10509 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:34.991375 10587 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:11:34.993628 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:34.994617 10434 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:34.965627 14988 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:35.003491 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:35.089409 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 86015.7us
I20260709 12:11:34.914834 14997 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.004520 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:34.991761 14914 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.005313 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:35.098446 10437 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:35.071779 15067 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:35.006430 15031 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.006943 14980 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.995091 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.023098 11221 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:35.025305 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 570) took 1509 ms. Trace:
I20260709 12:11:35.108249 11217 rpcz_store.cc:276] 0709 12:11:33.515455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.515620 (+   165us) service_pool.cc:224] Handling call
0709 12:11:33.518728 (+  3108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:34.002834 (+484106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.024974 (+1022140us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400591854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:35.025225 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057662848}
I20260709 12:11:35.027549 10552 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:35.031529 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62951us
I20260709 12:11:35.031648 15032 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:11:35.114857 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 87405.8us
I20260709 12:11:34.995122 10583 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:35.065609 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.116302 10586 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:35.066632 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.067093 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.067914 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.120100 10513 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:35.068073 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.120766 10510 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:35.068738 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.076076 10508 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:35.018132 14901 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.080971 11215 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:35.081200 10512 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:35.081336 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:35.050365 14515 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.158521 10511 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:35.159358 10429 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:35.092177 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 99174.1us
I20260709 12:11:35.090461 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:35.161290 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 70962.5us
I20260709 12:11:35.161635 11178 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:35.162127 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:35.072464 15001 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:35.095427 10432 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:35.100304 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2014) took 1045 ms. Trace:
I20260709 12:11:35.166317 10436 rpcz_store.cc:276] 0709 12:11:34.054492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.054646 (+   154us) service_pool.cc:224] Handling call
0709 12:11:34.054999 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:34.727249 (+672250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.100049 (+372800us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54002661d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:35.100239 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523735808}
I20260709 12:11:35.005945 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:35.169317 10581 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:35.169728 10581 raft_consensus.cc:614] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:35.077265 14837 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.112015 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:35.172593 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60732.8us
W20260709 12:11:35.107033 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 112048us
W20260709 12:11:35.175987 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:35.121207 10509 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:35.127981 14980 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.129109 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:35.183157 10513 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:35.130280 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.098498 14646 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:35.131517 14968 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:35.132241 15032 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:35.134024 14914 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:35.134182 15057 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:35.136322 14905 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:35.139760 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:35.140945 14959 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:35.144105 10437 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:35.144518 14987 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:35.145026 15070 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:35.151114 15031 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.153700 14997 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:35.098147 14972 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:35.095371 14930 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:35.156533 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.284124 10583 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:35.187227 15001 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:35.164350 11178 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:35.094376 14988 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.170079 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:35.306058 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 136052us
I20260709 12:11:35.308096 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.181006 10582 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:35.174038 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.185473 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.187705 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.310285 10580 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:35.181484 10586 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:35.311259 10506 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:11:35.251549 10585 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:35.172434 14910 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:35.165823 14974 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:35.247400 10404 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:35.172188 14566 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:35.328543 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 81341.8us
I20260709 12:11:35.268359 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.329876 10581 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:35.186149 14918 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:35.185431 15063 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:35.182395 14559 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:35.157867 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.238042 15057 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:35.316534 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 142563us
I20260709 12:11:35.283118 14987 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:35.281697 15055 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:35.321235 14569 raft_consensus.cc:2764] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:35.321640 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.326295 14988 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:35.322506 14515 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.266657 14905 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.186422 14837 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:35.323931 15045 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.703s	user 0.004s	sys 0.001s
I20260709 12:11:35.324127 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.367405 10586 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:35.334796 14997 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:35.324990 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.372673 10582 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:35.330488 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.339771 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:35.374473 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44101.4us
I20260709 12:11:35.327600 14901 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.334300 15061 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:35.342757 15055 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:35.348112 14810 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:35.349210 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 2021) took 1292 ms. Trace:
I20260709 12:11:35.350320 10587 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13.
I20260709 12:11:35.265960 14959 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.332124 14646 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:35.363248 10580 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:35.340914 10579 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:35.335561 15063 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:35.398707 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.397038 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.400920 10429 rpcz_store.cc:276] 0709 12:11:34.057092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.057359 (+   267us) service_pool.cc:224] Handling call
0709 12:11:35.348924 (+1291565us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540034a654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:35.349121 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2531510144}
I20260709 12:11:35.402760 10583 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:35.351490 14930 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:11:35.387815 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.245180 15067 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.425290 10586 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:35.442382 10581 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:35.443451 10586 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:35.455745 10581 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:35.467029 10579 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:35.471602 10512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:35.472360 10512 raft_consensus.cc:606] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:35.472733 10512 raft_consensus.cc:614] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:35.470068 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.474653 10579 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:11:35.377283 14918 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:35.345582 14559 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.411170 14994 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:35.489567 14980 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.489661 14978 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:35.490293 14978 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.490412 15057 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.423305 15059 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:11:35.424697 14810 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.491596 15057 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:35.493253 14810 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.489684 14994 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:35.397432 14910 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:35.377485 15037 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.495040 14959 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.410945 14893 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:11:35.495944 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2024) took 1405 ms. Trace:
I20260709 12:11:35.473973 14901 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:11:35.495138 15071 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.452966 15061 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:35.483762 14987 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.454778 14997 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:35.484678 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:35.485924 14665 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:35.486239 14905 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.486778  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:35.487112 15031 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.534334 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:35.487844 14837 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.488420 15001 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.489076 14559 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.495476 14918 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.495543 15037 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.476579 14569 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.481598 15063 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.496380 10438 rpcz_store.cc:276] 0709 12:11:34.090321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.090685 (+   364us) service_pool.cc:224] Handling call
0709 12:11:34.091111 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:34.124234 (+ 33123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:34.124301 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:34.124931 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:11:34.285433 (+160502us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:34.303989 (+ 18556us) log.cc:844] Serialized 14 byte log entry
0709 12:11:34.304329 (+   340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:34.304456 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:35.003586 (+699130us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b18002f3b00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:35.243822 (+240236us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:35.495704 (+251882us) raft_consensus.cc:1685] finished
0709 12:11:35.495728 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:35.495877 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29046,"prepare.queue_time_us":16052,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":58632,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":8621,"spinlock_wait_cycles":19296000,"thread_start_us":129513,"threads_started":1,"wal-append.queue_time_us":451388}
I20260709 12:11:35.536989 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:35.556413 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:35.594112 10511 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:35.556483 10585 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:35.532372  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:35.564558 10512 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:11:35.572539 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:35.495314 14910 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:35.557736 15075 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:11:35.596009 14910 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:35.501145 14646 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:35.505568 14868 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:35.603973 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67113.4us
I20260709 12:11:35.606542 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:35.598555 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 147001us
I20260709 12:11:35.562335 14893 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:35.606649 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.607528 10581 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:35.611883 10585 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:35.497144 15071 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.565656 15061 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:35.553781 14968 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:35.590643 14559 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:35.628340 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 94099.2us
I20260709 12:11:35.629887 10478 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13.
I20260709 12:11:35.630684 10478 leader_election.cc:403] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:35.602393 15031 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:35.495124 14994 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.616411 14893 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.591629 14837 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:11:35.636740 14971 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:35.625854 14810 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:35.597898 14569 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.634800 15031 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.636328 14930 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.629854 14559 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.639233 15087 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.615207 14868 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.635322 14997 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.650436 15031 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:35.653442 15070 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.663230 15070 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:35.657948 14930 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.649991 14646 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.667213 14930 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.655997 15037 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.653582 14810 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:35.642869 14893 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:35.533555 15055 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.637661 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:35.670871 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 506564us
W20260709 12:11:35.771675 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 134105us
I20260709 12:11:35.786751 10505 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:35.784740 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:35.787664 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:35.788630 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:35.789628 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:35.651886 14978 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:35.789894 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 176957us
I20260709 12:11:35.791932 10506 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:35.653378 14918 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.801759 10512 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:35.660396 14569 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.649883 15086 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.655128 15071 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:35.806003 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.810316 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.822460 10583 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:35.791352 15055 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:35.837893 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 450128us
I20260709 12:11:35.834621 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.826472 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:35.799656 14993 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:35.785004 15099 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:35.818265 15053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.021s	sys 0.007s
I20260709 12:11:35.838928 15097 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:35.651703 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55454
I20260709 12:11:35.846356 14810 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.861888 14868 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:35.862131 14559 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:35.862622 14868 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:35.864102 14868 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:35.849848 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.841709 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.653584 14968 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:35.664670 15087 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.910305 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:35.914997 11216 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:35.912423 10506 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:35.653851 15061 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:35.910240 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 116052us
W20260709 12:11:35.855474 15053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.021s	sys 0.007s
I20260709 12:11:35.850960 15099 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.935192 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.866964 15071 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:35.924433 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:35.916595 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 296993us
I20260709 12:11:35.792490 14837 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:11:35.945773 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:35.952960 10506 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:35.954618 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:35.932299 15046 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:11:35.836468 15086 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:35.957923 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:35.960995 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:35.962998 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:35.965533 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.966547 11213 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:35.970368 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:35.931408 15099 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:35.972645 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:35.979565 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:35.980437 10433 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:35.980726 10438 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:35.985387 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:35.986016 10431 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:35.944510 15071 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:35.994158 10431 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:35.971804 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:35.997237 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:35.997434 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:35.998708 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.999114 11216 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:35.999653 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:35.999825 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:35.997911 10438 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:36.000741 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:35.960738 15097 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:35.996313 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:36.007882 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:36.008188 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:36.001345 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.009730 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:35.981312 14993 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:36.011574 10438 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:35.981124 15101 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:35.981406 14918 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:36.023547 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:35.995460 15071 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.040905 10438 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:36.006206 15097 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:36.041491 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.042409 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:36.003309 14905 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 12:11:36.043937 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 78735.7us
I20260709 12:11:36.053741 15037 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:36.036885 15103 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:36.052636 15110 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:36.073354 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:36.046738 14980 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:11:36.051358 14559 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:11:36.072518 14969 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:36.109983 10433 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:36.066629 15005 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:11:36.076510 15037 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.060570 15001 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:36.099272 15092 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:36.068245 14515 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:11:36.080869 15110 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.076341 15094 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:36.151587 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 131818us
I20260709 12:11:36.075094 15070 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:36.152156 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 305181us
W20260709 12:11:36.159637 15057 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.668s	user 0.015s	sys 0.000s
W20260709 12:11:36.161989 15109 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:11:36.161906 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:36.162932 10513 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:36.163053 15098 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:11:36.163352 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.110718 14978 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:11:36.166461 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.116000 14969 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:36.157886 15112 tablet_replica.cc:333] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:11:36.131076 15037 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:36.130295 14646 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:36.155306 15095 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.226s	user 0.002s	sys 0.000s
I20260709 12:11:36.140089 15092 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.175810 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:36.176542 11216 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:36.169795 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.177515 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.179442 10509 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:36.181703 10511 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:36.181945 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:36.183715 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:36.156708 15110 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.191182 15124 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.194607 15124 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:36.121356 14993 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.197182 14893 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.197847 15124 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:36.143278 15001 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:36.205163 15001 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:36.169780 15094 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:36.208384 15094 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:36.209343 14969 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:36.210799 14969 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:36.211308 15086 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:36.212111 14893 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:36.215587 10586 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:36.179826 15092 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.111285 15097 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.229333 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.231261 15057 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:36.262715 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.298578 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:36.303187 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 164389us
I20260709 12:11:36.306386 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:36.310650 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.347337 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:36.351013 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 169188us
I20260709 12:11:36.385648 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:36.186333 15087 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:36.407033 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 283892us
I20260709 12:11:36.181313 14646 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.346179 15032 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:36.348590 15057 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:36.400957 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:36.425843 10384 auto_rebalancer.cc:629] Failed to schedule move for tablet c5a50eeaf1ac462883e185eddec889f6: Illegal state: Replica 99d42e7288f145f1bd5af212fd062995 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:36.349000 15091 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:36.424758 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:36.391875 14978 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:36.280817 14910 consensus_meta.cc:287] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.684s	user 0.009s	sys 0.005s
I20260709 12:11:36.393627 14980 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:36.391920 15114 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:36.394436 15059 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:36.440176 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:36.441679 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.454347 10508 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:36.448032 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:36.448160 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:36.456341 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 281430us
I20260709 12:11:36.457340 10580 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:36.462212 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:36.470570 10581 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:36.440665 14988 consensus_meta.cc:287] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.068s	user 0.016s	sys 0.000s
I20260709 12:11:36.444262 14978 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:36.483006 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:36.493208 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53180.9us
I20260709 12:11:36.442129 15091 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:36.497977 15091 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.507903 10581 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:36.509473 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:36.515079 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:36.516948 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:36.433979 15061 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:36.520510 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.522997 15125 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:36.484242 15055 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.632s	user 0.015s	sys 0.000s
I20260709 12:11:36.528836 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:36.530889 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 115736us
I20260709 12:11:36.493913 14978 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:36.453687 14910 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:36.532769 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.541838 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:36.513640 15094 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:36.488597 14988 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.561309 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.563467 10512 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:36.572168 10508 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:36.545778 15070 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:36.575259 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60547us
I20260709 12:11:36.662178 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:36.668550 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.530179 15061 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.537101 15086 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 12:11:36.663026 15031 consensus_meta.cc:287] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.000s	user 0.013s	sys 0.002s
I20260709 12:11:36.704038 15129 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:36.665316 15085 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:36.677734 14971 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.036s	user 0.013s	sys 0.003s
I20260709 12:11:36.731696 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:36.570838 15094 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 12:11:36.704056 15031 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:36.747247 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 292392us
I20260709 12:11:36.737542 15096 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:36.737628 14971 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13.
I20260709 12:11:36.732638 15127 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:36.742558 15031 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:36.771431 14959 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.778056 14646 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:36.787678 15099 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.783s	user 0.010s	sys 0.004s
I20260709 12:11:36.781854 15102 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:36.798519 14559 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:36.820837 14988 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:36.827999 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.828408 14910 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:36.797640 14959 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.829025 15144 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:36.829707 10505 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:36.832921 14795 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:11:36.812764 15099 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:36.839208 14905 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:36.846593 15122 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:11:36.840605 14959 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:36.820912 14559 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:36.868741 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 342728us
I20260709 12:11:36.856194 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2636) took 1696 ms. Trace:
I20260709 12:11:36.946534 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:36.947649 10508 raft_consensus.cc:2368] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:36.946600 10578 rpcz_store.cc:276] 0709 12:11:35.159627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.159763 (+   136us) service_pool.cc:224] Handling call
0709 12:11:35.160085 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:35.477342 (+317257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:35.477392 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:35.477451 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:35.477495 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.855874 (+1378379us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b540010b8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:36.855935 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:36.856113 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729021696}
W20260709 12:11:36.856475 14969 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.001s
W20260709 12:11:36.837404 14993 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.002s	sys 0.012s
W20260709 12:11:36.955842 14993 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.004s	sys 0.012s
I20260709 12:11:36.953867 14914 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:36.956766 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:36.957099 14914 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:36.959833 10510 raft_consensus.cc:1217] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Deduplicated request from leader. Original: 7.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 12:11:36.960263 15057 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.960784 15103 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:36.961248 14918 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:36.961339 15035 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:36.961719 15017 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:36.961966 14988 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:36.956180 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 86203.3us
I20260709 12:11:36.963527 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.002868 10436 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:37.004299 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:36.870431 15110 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.003s
I20260709 12:11:36.966804 10510 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:36.968894 10435 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:36.977988 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:36.983978 10508 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:36.987370 10433 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:37.011410 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:36.840144 15136 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:37.001849 15017 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.015964 15136 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.016160 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:36.950634 15126 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.352s	user 0.002s	sys 0.000s
I20260709 12:11:37.018026 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:37.021896 10432 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.845767 15094 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.855274 15061 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.005053 10436 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:37.015268 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.051182 14993 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.840821 15138 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.028667 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.022663 15017 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.022840 15060 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:37.055847 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.056279 15060 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.058259 14795 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.059130 14795 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.059837 15138 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.061815 14795 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.062419 14795 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:37.063503 15148 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:11:37.058825 15060 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:37.069622 15037 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.886s	user 0.012s	sys 0.000s
I20260709 12:11:37.006044 14988 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:37.039770 15145 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:11:36.990016 15057 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.040037 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 480) took 2316 ms. Trace:
I20260709 12:11:36.995945 14918 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.003389 15035 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.049220 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2182) took 1269 ms. Trace:
I20260709 12:11:37.052790 10435 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:37.073660 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:36.993891 15103 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.031509 14978 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.075408 15058 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.018900 15110 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.076125 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2063) took 1090 ms. Trace:
I20260709 12:11:37.076385 15136 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.077688 15099 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:37.078665 15111 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.953s	user 0.009s	sys 0.002s
W20260709 12:11:37.079871 15092 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.863s	user 0.014s	sys 0.002s
W20260709 12:11:37.033989 15139 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:11:37.090374 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:37.186677 10433 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:37.123286 10579 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:37.159346 10403 leader_election.cc:400] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:37.187533 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.187834 10403 leader_election.cc:403] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:11:37.188294 10435 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 8.
I20260709 12:11:37.074860 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2642) took 1820 ms. Trace:
W20260709 12:11:37.148952 10511 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.050s	sys 0.059s
W20260709 12:11:37.188448 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56648.2us
W20260709 12:11:37.189034 10511 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.052s	sys 0.059s
I20260709 12:11:37.160956 10434 rpcz_store.cc:276] 0709 12:11:34.724935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.086797 (+361862us) service_pool.cc:224] Handling call
0709 12:11:35.087212 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:35.106210 (+ 18998us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:35.129107 (+ 22897us) log.cc:844] Serialized 14 byte log entry
0709 12:11:35.531980 (+402873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:35.532040 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:35.532089 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.039695 (+1507606us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5400264dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.039775 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.039951 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27151,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":24457,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":26713,"spinlock_wait_cycles":42985216,"thread_start_us":401590,"threads_started":1,"wal-append.queue_time_us":510101,"wal-append.run_cpu_time_us":1466,"wal-append.run_wall_time_us":1012278}
I20260709 12:11:37.175735 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:37.188438 10432 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:37.179829 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.180876 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:37.161585 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 88057.2us
I20260709 12:11:37.192605 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.192946 15055 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:37.193562 10580 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:37.193805 15055 consensus_queue.cc:1243] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:37.193913 15057 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.194365 15055 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.186022 15099 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:37.195613 15099 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.197978 15099 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.159925 15061 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.043708 15144 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:37.177282 10438 rpcz_store.cc:276] 0709 12:11:35.986060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.047830 (+ 61770us) service_pool.cc:224] Handling call
0709 12:11:36.048111 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.056392 (+  8281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.056442 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:36.056520 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:36.056566 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.075777 (+1019211us) spinlock_profiling.cc:243] Waited 8.03 ms on lock 0x7b5400437e54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.075846 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.076036 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18470272}
I20260709 12:11:37.160862 10507 rpcz_store.cc:276] 0709 12:11:35.780106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.780247 (+   141us) service_pool.cc:224] Handling call
0709 12:11:35.780695 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:35.780947 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:11:35.797894 (+ 16947us) log.cc:844] Serialized 14 byte log entry
0709 12:11:36.002086 (+204192us) log.cc:844] Serialized 14 byte log entry
0709 12:11:36.978159 (+976073us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b180038b3c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c92ad7060 00007f2c92ae9716 00007f2c92afa287 00007f2c92afa208
0709 12:11:37.047448 (+ 69289us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:37.047530 (+    82us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:11:37.047597 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.047683 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.049126 (+  1443us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7194,"prepare.queue_time_us":17585,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":107,"raft.queue_time_us":228799,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":7816,"spinlock_wait_cycles":8670080,"thread_start_us":266180,"threads_started":2,"wal-append.queue_time_us":38010,"wal-append.run_cpu_time_us":3022,"wal-append.run_wall_time_us":1228763}
I20260709 12:11:37.127496 15150 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:37.127691 15071 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.072s	user 0.016s	sys 0.000s
I20260709 12:11:37.188892 10582 rpcz_store.cc:276] 0709 12:11:35.254737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.394603 (+139866us) service_pool.cc:224] Handling call
0709 12:11:35.395031 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:35.495594 (+100563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:35.495629 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:35.495689 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:35.495727 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.074447 (+1578720us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b540010b3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.074542 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.074764 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230733568}
I20260709 12:11:37.213668 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.214609 10582 raft_consensus.cc:2410] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:11:37.201165 10512 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:37.194583 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.091079 14795 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.191054 10586 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:37.092712 14984 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:37.161340 15102 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:37.091310 15037 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.205742 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.202075 15017 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.159686 15060 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:37.208020 10581 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:37.224925 10432 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:37.204092 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:37.222808 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.221741 10512 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:37.223110 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.194608 15057 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.052531 14993 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.232738 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.053972 15094 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.236665 10580 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:37.236977 10509 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
W20260709 12:11:37.236991 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.232429 15060 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.240572 14988 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.241978 15035 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.194904 15055 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.179709 15058 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.200930 15092 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.250454 14795 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.214334 15061 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.217542 15086 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:11:37.273569 14918 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.260444 14559 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.263870 15060 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.224903 15071 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.226032 14984 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.218529 15144 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.212606 15150 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.235138 15057 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.267917 15067 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:37.260893 15037 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.289877 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.289810 10432 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:37.297657 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.301122 14984 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.306134 10578 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:37.307765 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.308790 10578 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 16 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:11:37.312038 10580 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:11:37.312634 11219 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:37.314533 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.317409 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:37.323241 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.328730 10504 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:11:37.334095 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.325399 14918 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:37.342949 15094 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.344301 15150 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.345723 15097 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.327751 14984 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:37.339886 14918 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:37.341204 14978 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.357705 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.346828 15110 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.363528 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.364575 10578 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:37.368863 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.369815 10578 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:37.375092 15057 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.362135 14918 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:37.379259 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.391510 15071 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.352602 14984 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.393651 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:37.396613 11217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 657) took 2256 ms (client timeout 2474 ms). Trace:
I20260709 12:11:37.396919  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:37.397070 11217 rpcz_store.cc:269] 0709 12:11:35.139197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.139412 (+   215us) service_pool.cc:224] Handling call
0709 12:11:37.395021 (+2255609us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b5400791154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:37.395207 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1853861632}
W20260709 12:11:37.399751 15132 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.844s	user 0.012s	sys 0.008s
I20260709 12:11:37.400779 14984 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.403723 15063 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:37.405879 15142 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.671s	user 0.002s	sys 0.000s
I20260709 12:11:37.422510 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2471) took 1953 ms. Trace:
I20260709 12:11:37.422886 10579 rpcz_store.cc:276] 0709 12:11:35.469213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.488026 (+ 18813us) service_pool.cc:224] Handling call
0709 12:11:35.488350 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:37.187807 (+1699457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.422184 (+234377us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400336654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.422439 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3758907776}
I20260709 12:11:37.380052 14993 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.414294 10578 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:37.419826 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 584) took 1596 ms. Trace:
I20260709 12:11:37.426012 11221 rpcz_store.cc:276] 0709 12:11:35.823123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.824362 (+  1239us) service_pool.cc:224] Handling call
0709 12:11:35.824654 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.846744 (+1022090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.846800 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:36.846853 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:36.846897 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.419543 (+572646us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400593154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.419596 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.419758 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2349998976}
I20260709 12:11:37.425064 15092 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.449302 14993 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.459833 14918 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:37.461465 15146 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.586s	user 0.002s	sys 0.000s
I20260709 12:11:37.467985 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2207) took 1041 ms. Trace:
I20260709 12:11:37.468361 10513 rpcz_store.cc:276] 0709 12:11:36.426152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.426301 (+   149us) service_pool.cc:224] Handling call
0709 12:11:36.426650 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:36.442956 (+ 16306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.443028 (+    72us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:36.443817 (+   789us) log.cc:844] Serialized 52 byte log entry
0709 12:11:36.500218 (+ 56401us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:36.500283 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.859252 (+358969us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b5400360454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:36.859314 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.405231 (+545917us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:37.467725 (+ 62494us) raft_consensus.cc:1685] finished
0709 12:11:37.467748 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.467925 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":666,"raft.queue_time_us":2754,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":9122,"spinlock_wait_cycles":43503104,"thread_start_us":11117,"threads_started":1,"wal-append.queue_time_us":202760}
I20260709 12:11:37.463250 14993 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.475440 15112 raft_consensus.cc:2243] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 12:11:37.466879 15144 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.486035 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2055) took 1502 ms. Trace:
I20260709 12:11:37.486483 10429 rpcz_store.cc:276] 0709 12:11:35.983380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.983599 (+   219us) service_pool.cc:224] Handling call
0709 12:11:35.983998 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.055470 (+ 71472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.055510 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:36.055555 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:36.055598 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.485748 (+1430150us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b5400166cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.485804 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.485959 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163840384}
I20260709 12:11:37.479172 15110 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:37.493259 15110 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.489742 15032 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.496675 15110 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.499650 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2659) took 1045 ms. Trace:
I20260709 12:11:37.500489 10585 rpcz_store.cc:276] 0709 12:11:36.453839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.474449 (+ 20610us) service_pool.cc:224] Handling call
0709 12:11:36.474889 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:36.484332 (+  9443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.484370 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:36.513941 (+ 29571us) log.cc:844] Serialized 26 byte log entry
0709 12:11:36.773245 (+259304us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:36.773297 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.773382 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.460795 (+687413us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:37.499291 (+ 38496us) raft_consensus.cc:1685] finished
0709 12:11:37.499318 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.499580 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8810,"raft.queue_time_us":77036,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":19950,"spinlock_wait_cycles":95672704,"thread_start_us":217661,"threads_started":1,"wal-append.queue_time_us":285389}
I20260709 12:11:37.497714 15032 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.497956 15112 raft_consensus.cc:2272] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:11:37.502380 15017 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:37.503614 10431 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:11:37.507261 15147 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.564s	user 0.002s	sys 0.000s
I20260709 12:11:37.507582 14993 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.508891 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2654) took 1062 ms. Trace:
I20260709 12:11:37.509296 10584 rpcz_store.cc:276] 0709 12:11:36.446792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.447628 (+   836us) service_pool.cc:224] Handling call
0709 12:11:36.476608 (+ 28980us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:36.612855 (+136247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.612910 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:36.613819 (+   909us) log.cc:844] Serialized 52 byte log entry
0709 12:11:36.793636 (+179817us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:36.793701 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.796683 (+  2982us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.506545 (+709862us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:37.508541 (+  1996us) raft_consensus.cc:1685] finished
0709 12:11:37.508568 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.508821 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13871,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":19431,"spinlock_wait_cycles":95336832,"thread_start_us":270064,"threads_started":2,"wal-append.queue_time_us":213442}
I20260709 12:11:37.511145 10583 raft_consensus.cc:1217] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 12:11:37.511677 10583 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:37.518155 10510 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:37.520373 14918 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.506800 15112 ts_tablet_manager.cc:1916] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:11:37.523518 15058 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:37.525975 10505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 2189) took 1736 ms (client timeout 2052 ms). Trace:
I20260709 12:11:37.527910 14994 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.528559 14994 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.530045 10437 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:37.531420 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2578) took 2583 ms. Trace:
I20260709 12:11:37.531778 10437 rpcz_store.cc:276] 0709 12:11:34.948315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.301230 (+352915us) service_pool.cc:224] Handling call
0709 12:11:35.301700 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:37.530409 (+2228709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.531150 (+   741us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b540033dbd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.531355 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5123806464}
I20260709 12:11:37.535404 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.536185 10504 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:37.537182 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.538355 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:37.540448 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:37.540633 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2479) took 1118 ms. Trace:
I20260709 12:11:37.541100 10509 rpcz_store.cc:276] 0709 12:11:36.422677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.423070 (+   393us) service_pool.cc:224] Handling call
0709 12:11:36.438146 (+ 15076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:37.246182 (+808036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.540352 (+294170us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54002c6fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.540563 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836458880}
I20260709 12:11:37.541791 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.542539 10506 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:37.545382 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.551023 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 481) took 2825 ms. Trace:
I20260709 12:11:37.552047 10504 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:37.553262 15063 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:37.553882 15063 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.556332 15063 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.557931 15063 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:37.559496 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:37.526043 15097 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:11:37.560277 15097 consensus_queue.cc:1243] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:37.558483 15063 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.563382 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:37.563845 15063 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.564927 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:37.566355 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:37.567379 11213 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:11:37.569197 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.569934 10437 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:11:37.570528 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.571143 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.571738 10434 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:11:37.572170 10434 pending_rounds.cc:85] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 9
I20260709 12:11:37.572412 10434 pending_rounds.cc:107] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:11:37.572655 10434 raft_consensus.cc:2889] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:37.582199 15063 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:37.583261 15063 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.587500 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:37.551517 10430 rpcz_store.cc:276] 0709 12:11:34.725158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.099041 (+373883us) service_pool.cc:224] Handling call
0709 12:11:35.099651 (+   610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:35.099897 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:35.099927 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:35.100468 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:11:35.593639 (+493171us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:35.613545 (+ 19906us) log.cc:844] Serialized 14 byte log entry
0709 12:11:35.804763 (+191218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.804852 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.544839 (+1739987us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b180015a240. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:37.550661 (+  5822us) raft_consensus.cc:1685] finished
0709 12:11:37.550783 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.550931 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23148,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4608,"raft.queue_time_us":34986,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":7155,"spinlock_wait_cycles":20298624,"thread_start_us":374610,"threads_started":2,"wal-append.queue_time_us":466638}
W20260709 12:11:37.526494 10505 rpcz_store.cc:269] 0709 12:11:35.789213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.789350 (+   137us) service_pool.cc:224] Handling call
0709 12:11:37.525766 (+1736416us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54000d9b54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:37.525892 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308010880}
I20260709 12:11:37.596109 10429 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:37.597756 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:37.597519 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:37.629280 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:37.629884 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:37.631927 10587 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:37.632321 10582 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:37.621572 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:37.639385 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:37.644119 10581 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:37.645534 15110 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:37.646330 15110 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:37.647639 15110 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.652988 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:37.653550 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51461.1us
I20260709 12:11:37.654605 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:37.656478 10584 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.656683 10582 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:37.657543 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:37.651104 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53413.2us
I20260709 12:11:37.626801 15154 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:11:37.648625 15087 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:37.714752 15158 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.717267 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:37.668110 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:37.600041 14918 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:11:37.633873 15131 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:37.723518 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:37.760586 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 105723us
W20260709 12:11:37.755483 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 87494.7us
I20260709 12:11:37.776223 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.776576 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.778632 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:37.768564 14918 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:37.760818 15093 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:37.761473 15158 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.699084 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:37.764243 15087 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:37.764473 15063 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.787007 10581 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:37.760154 15060 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:37.769904 14810 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:37.794757 10509 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:37.792682 14810 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:37.782102 14918 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:37.786875 15093 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:37.769989 15131 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.796191 10404 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:37.797883 14810 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:37.836031 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67935.5us
I20260709 12:11:37.812152 15093 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.845417 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:37.846437 11215 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:37.852387 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:37.905596 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53354.3us
W20260709 12:11:37.907382 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 145915us
W20260709 12:11:37.908882 15170 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:11:37.909207 15158 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.835283 15060 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:11:37.915122 15063 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:37.627360 15160 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:37.655758 15058 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:37.936713 15055 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:37.844528 14918 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.903538 15087 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.903334 14980 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.944655 14914 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:37.950539 15063 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:37.955638 10584 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:37.906534 15094 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.950958 15160 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:37.963016 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57166.8us
I20260709 12:11:37.976037 15131 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:37.967439 14515 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:11:37.977180 10513 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:37.979673 10584 raft_consensus.cc:1217] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 9.7->[11.8-15.10]   Dedup: 11.8->[13.9-15.10]
I20260709 12:11:37.980433 10584 pending_rounds.cc:85] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 8
I20260709 12:11:37.980906 15160 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:37.980968 10584 pending_rounds.cc:107] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:11:37.982331 10584 raft_consensus.cc:2889] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:37.959925 15058 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:37.987510 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:37.961391 15063 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:37.989498 15058 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.968158 14918 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:37.961160 14980 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:37.993094 14515 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:37.966375 15094 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:37.993911 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33248 (request call id 5) took 2072 ms. Trace:
I20260709 12:11:38.040999 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:38.049597 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:38.050702 15060 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:38.036720 11214 rpcz_store.cc:276] 0709 12:11:35.913307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.915471 (+  2164us) service_pool.cc:224] Handling call
0709 12:11:37.961388 (+2045917us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b5400830754. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae5735 00007f2c92add9fa 00007f2c97aa54c9 00007f2c91021c54 00007f2c91021be2 00007f2c91021b11 00007f2c91021a8c 00007f2c91020d42 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:37.983550 (+ 22162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123270016}
W20260709 12:11:38.053260 10323 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e4cd5d330f694dacb199bd61266c8f5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 52bae2d2caf94d97b1049ecd4f82a2df is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:38.004740 15131 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:11:38.158651 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2070) took 2170 ms. Trace:
I20260709 12:11:38.159348 10431 rpcz_store.cc:276] 0709 12:11:35.987975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.119589 (+131614us) service_pool.cc:224] Handling call
0709 12:11:36.119993 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:37.503969 (+1383976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.158309 (+654340us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540034ab54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:38.158568 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3181174016}
W20260709 12:11:38.161954 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 127860us
I20260709 12:11:38.063629 15158 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:38.042863 14515 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.081977 14980 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:38.159093 15155 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:11:38.079941 15094 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:38.066783 14914 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:11:38.087131 15060 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 12:11:38.221791 15059 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:38.226334 15059 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:38.227361 15059 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.213451 14810 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:38.224104 15157 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.101s	user 0.011s	sys 0.009s
W20260709 12:11:38.244810 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 78834.9us
I20260709 12:11:38.232775 15103 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:38.259699 15189 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:11:38.286286 15192 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:11:38.295395 15138 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.341609 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.342690 10504 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:38.345052 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.346343 10504 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:11:38.357937 10384 auto_rebalancer.cc:629] Failed to schedule move for tablet 945dd7b821a5493f8a9d6ca251795da9: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:11:38.361447 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.361668 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:38.363466 15017 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.366245 14972 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.366946 14972 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.364032 15158 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.368503 15158 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:38.369257 14972 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.359521 14566 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.371587 14566 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:38.356115 15176 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.163s	user 0.000s	sys 0.009s
I20260709 12:11:38.356130 15087 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.373814 14566 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.374177 15087 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.376338 15087 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:38.376543 11219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57904 (request call id 617) took 2435 ms (client timeout 1536 ms). Trace:
W20260709 12:11:38.377449 15184 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:11:38.379532 15138 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:38.380054 15138 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:38.380445 15138 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:38.384378 15061 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.386521 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.388409 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:38.389344 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:38.389837 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.390465 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:38.391669 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2485) took 1295 ms. Trace:
I20260709 12:11:38.392026 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:38.392105 10510 rpcz_store.cc:276] 0709 12:11:37.096308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.175116 (+ 78808us) service_pool.cc:224] Handling call
0709 12:11:38.391495 (+1216379us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54000da554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:38.391604 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752834816}
I20260709 12:11:38.393292 15128 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:38.393429 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:38.393772 15128 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.394230 15128 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:38.391683 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:38.397706 10511 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:38.398753 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.399731 10510 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:11:38.400512 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:38.377068 11219 rpcz_store.cc:269] 0709 12:11:35.941574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.941754 (+   180us) service_pool.cc:224] Handling call
0709 12:11:38.376298 (+2434544us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540052f3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:38.376458 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3096702848}
I20260709 12:11:38.401733 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.402375 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.401762 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:38.403411 10586 raft_consensus.cc:2368] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:38.403330 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.403650 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.366034 15017 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.405869 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.406334 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.407671 10403 leader_election.cc:400] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:38.408018 10403 leader_election.cc:403] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:11:38.408769 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.410300 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.410239 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:38.414626 10507 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:38.415097 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.341368 14905 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.416520 10504 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:38.416631 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:38.417748 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 172120us
I20260709 12:11:38.419597 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.420264 10433 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:38.421967 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.426501 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:38.426683 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.427345 10438 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:38.429072 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:38.429728 10431 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:38.429919 10438 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:38.415968 14979 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.434163 10507 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:38.436000 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.439519 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.439802 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.440109 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.440524 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.441202 10437 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:38.440820 10433 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:38.442742 10431 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:38.456636 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:38.482653 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2075) took 1428 ms. Trace:
I20260709 12:11:38.483428 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.483100 10432 rpcz_store.cc:276] 0709 12:11:37.053890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.230421 (+176531us) service_pool.cc:224] Handling call
0709 12:11:37.254767 (+ 24346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:37.295990 (+ 41223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.482418 (+1186428us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54003962d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:38.482591 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54513920}
I20260709 12:11:38.484889 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.485781 10581 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:11:38.486333 10431 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:38.480024 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:38.487684 10580 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:38.435043 15123 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:11:38.481665 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.491087 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.501880 10438 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:38.417528 14559 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.416241 15061 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.512706 10429 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:38.417553 15087 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:38.521493 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 84214.7us
I20260709 12:11:38.436981 14914 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:38.421264 14905 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:38.434620 14979 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:38.526517 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 76508.3us
I20260709 12:11:38.442034 15158 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.523031 14515 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.562014 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:38.570211 10432 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:11:38.573045 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50470us
W20260709 12:11:38.576491 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.525095 14559 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.513196 15061 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.582450 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.558144 15112 ts_tablet_manager.cc:1929] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 7.5
I20260709 12:11:38.586719 15032 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.593652 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:38.602849 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 175507us
I20260709 12:11:38.594050 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:38.527495 15087 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.567268 14810 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:38.609966 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 130378us
I20260709 12:11:38.604004 10508 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:38.615931 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.619347 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:38.620230 10429 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:38.625779 10438 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:38.641037 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.591391 14566 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.647686 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:38.643740 15005 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:11:38.651252 14566 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.654461 14566 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.663895 15017 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.665660 14559 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.671468 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:38.675856 15063 consensus_meta.cc:287] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.004s
I20260709 12:11:38.675945 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.676468 15063 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:38.680677 15160 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.007s
I20260709 12:11:38.681161 15160 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.683187 10430 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:38.614318 15093 consensus_meta.cc:287] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.711s	user 0.016s	sys 0.000s
I20260709 12:11:38.687115 15093 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.689349 15093 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:38.621510 15191 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.371s	user 0.000s	sys 0.002s
I20260709 12:11:38.700471 15061 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:38.707749 15200 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:11:38.644618 15112 log.cc:1199] T 0b474d9b241d4a698702287ee1bbe3b2 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/0b474d9b241d4a698702287ee1bbe3b2
I20260709 12:11:38.711462 15112 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55454 (request call id 73) took 3248 ms. Trace:
I20260709 12:11:38.711962 15112 rpcz_store.cc:276] 0709 12:11:35.497154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.585198 (+ 88044us) service_pool.cc:224] Handling call
0709 12:11:36.119745 (+534547us) ts_tablet_manager.cc:1151] Deleting tablet 0b474d9b241d4a698702287ee1bbe3b2
0709 12:11:36.142562 (+ 22817us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:11:37.998383 (+1855821us) tablet_metadata.cc:665] Metadata flushed
0709 12:11:38.536911 (+538528us) tablet_metadata.cc:665] Metadata flushed
0709 12:11:38.711328 (+174417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1244079,"tablet-delete.queue_time_us":217544,"thread_start_us":126172,"threads_started":1}
W20260709 12:11:38.719668 15058 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.005s
I20260709 12:11:38.720326 15058 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:38.724490 15201 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:11:38.737075 10435 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:38.726691 15087 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.729182 14979 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.591372 14901 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:11:38.618711 14997 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 12:11:38.731536 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:38.723233 10436 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:38.735190 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:38.745286 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 209352us
I20260709 12:11:38.747169 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:38.748610 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:38.749044 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:38.744776 10437 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:38.752045 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:38.765412 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 161434us
I20260709 12:11:38.749965 14901 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:38.750295 15061 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.767654 14979 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:38.779258 15206 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:38.779845 14979 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:38.780151 15206 consensus_queue.cc:1243] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:38.764981 15094 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:38.781852 15206 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.782418 15206 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:38.783542 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 169765us
I20260709 12:11:38.779814 15061 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.771137 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:38.785847 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:38.797678 10508 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:38.750769 15063 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:38.768821 15058 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.800652 15205 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:38.797204 15138 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.812012 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:38.807924 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:38.786360 14979 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:38.766188 14566 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:11:38.797629 15202 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:38.819180 10587 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:38.820152 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:38.825340 10586 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:38.805670 15160 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.827029 15202 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:38.836949 15202 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.808991 15205 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.842494 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:38.849107 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:38.846072 14901 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:38.843256 15196 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:38.853482 15196 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:38.854274 15208 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:11:38.854588 15196 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.858457 14901 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:38.817281 15092 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:38.803346 15197 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:11:38.859516 15138 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:38.863269 15207 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:38.871766 15207 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:38.872270 15207 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:38.867308 15206 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.829063 15199 raft_consensus.cc:2764] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:38.839001 15202 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:38.860419 15205 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:38.898900 15092 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.902372 15092 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:38.862866 15094 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:38.903483 15094 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.904036 15094 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:38.927654 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 328442us
W20260709 12:11:38.929437 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 150851us
I20260709 12:11:38.943398 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:38.944334 11212 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:38.943436 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:38.949476 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:38.956979 15058 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.957801 15058 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.960723 15058 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.963414 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:38.967061 15210 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.967947 15210 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.969432 15017 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:38.970166 15017 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.972870 15103 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.973632 15103 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:38.975205 15093 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.975983 15093 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:38.950874 11212 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:38.980741 11213 raft_consensus.cc:2376] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15. Re-sending same reply.
I20260709 12:11:38.987671 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:38.988648 11215 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:38.978745 15093 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:38.948863 15087 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:38.996034 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 147040us
I20260709 12:11:38.996996 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:38.984392 14914 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 9. Current term is 10.
I20260709 12:11:39.000639 10404 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:38.982887 15058 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:39.013098 10429 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:39.010694 15017 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.002370 15092 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:39.016688 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:39.019479 11220 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:39.009544 15160 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:39.005576 15138 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:11:39.016727 15213 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:38.999120 15087 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:39.042248 10479 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.68:40987 timed out after 1.737s (SENT) [suppressed 15 similar messages]
I20260709 12:11:39.022431 15061 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:38.494179  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:38.721448 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 0b474d9b241d4a698702287ee1bbe3b2 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:11:39.049321 15212 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:11:39.012696 15058 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:39.051795 10322 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 330551us
I20260709 12:11:39.032380 15160 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:39.017937 14905 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:39.038841 15128 consensus_meta.cc:287] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.008s
I20260709 12:11:39.065598 10584 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:39.067255 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:39.054826 15103 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.076152 14905 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:39.055606 15160 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:39.076370 15058 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.082543 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.083240 10510 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:11:39.080857 15210 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.057014 15196 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:11:39.095602 11220 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:11:39.095942 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.080787 15061 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.079064 14515 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:39.089277 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.061661 15197 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 12:11:39.110206 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 85325.7us
I20260709 12:11:39.061617 15128 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:39.101024 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 58790.6us
I20260709 12:11:39.102155 10511 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:39.050630 15213 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.119007 14994 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:39.119328 15216 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:11:39.122793 10510 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:11:39.131992 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:39.134135 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:39.114563 14515 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:39.138768 11219 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:39.139514 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.122056 15197 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:39.151527 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.112215 15061 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.131399 15053 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:39.160037 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:39.161616 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:39.162292 10431 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:39.162758 10436 pending_rounds.cc:85] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 6
I20260709 12:11:39.163208 10436 pending_rounds.cc:107] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:11:39.163244 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:39.163669 10436 raft_consensus.cc:2889] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:39.164044 10438 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:39.165184 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:39.166015 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:39.166010 10433 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:39.166647 10438 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:39.139755 15178 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:39.157560 10430 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:39.136184 15198 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.167346 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:39.167228 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:39.168361 10431 pending_rounds.cc:85] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 5
I20260709 12:11:39.176178 10431 pending_rounds.cc:107] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:11:39.176553 10431 raft_consensus.cc:2889] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:39.175545 10433 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:39.176805 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:39.181035 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:39.178682 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:39.154529 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:39.158372 15058 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.168350 14997 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:39.198319 10438 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:39.182911 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:39.175736 15178 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:39.182552 15093 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:39.188597 14972 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:39.158977 15197 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.189426 15199 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:39.207739 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53285.9us
I20260709 12:11:39.213707 15061 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:39.209273 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55916.4us
I20260709 12:11:39.217581 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.169526 15053 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:39.211134 15219 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:11:39.208353 15178 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:39.209365 14972 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:39.202971 14979 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.181355 15198 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:39.233331 15215 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:11:39.204229 15058 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.228032 14997 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:39.247669 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.235916 14972 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:39.249145 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.227859 15231 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:11:39.256984 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.233690 15230 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.236452 15199 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:11:39.227705 15229 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:11:39.209782 14515 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.271068 15128 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.297537 15221 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.299082 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:39.299883 14905 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:39.303916 15218 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.240s	user 0.003s	sys 0.000s
I20260709 12:11:39.304736 15223 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.308563 15053 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:39.306782 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 120962us
I20260709 12:11:39.293224 15087 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.339250 14969 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:11:39.341384 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 212364us
I20260709 12:11:39.363818 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:39.347465 15197 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:39.365919 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 146704us
I20260709 12:11:39.325743 15094 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.374449 15094 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:39.342329 15221 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.375234 10582 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:39.376690 15221 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.378082 15198 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.379076 10479 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.387287 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.387439 15238 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:39.388015 15238 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:39.388412 15238 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:39.389420 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.391474 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.393272 15197 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:39.393829 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.395277 10580 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:39.343130 14905 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.395949 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.387446 11214 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:39.382259 11220 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:39.385613 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.404410 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.411785 10580 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:39.369432 14969 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:39.414947 14969 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.415956 14979 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.420692 15186 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:39.421169 15186 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:39.422051 15186 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.424940 15059 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:39.350850 15223 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.432726 14980 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.433324 14980 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.435461 15058 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.435714 14980 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.437497 14905 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:39.437529 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:39.440564 15205 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.545s	user 0.014s	sys 0.000s
I20260709 12:11:39.431900 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.442394 10580 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:39.428002 15207 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.555s	user 0.015s	sys 0.002s
I20260709 12:11:39.458715 15207 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.451539 14901 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.462625 14901 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.454316 15186 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:39.472205 15223 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:39.472528 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 291919us
I20260709 12:11:39.473452 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:39.475198 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:39.470773 11212 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:39.478847 10510 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:11:39.478716 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:39.480662 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:39.481547 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:39.482071 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71820.8us
W20260709 12:11:39.488008 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 240522us
I20260709 12:11:39.488193 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.492233 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.496572 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:39.496783 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.491575 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.498927 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.499171 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.498283 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.498116 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:39.501101 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 129994us
I20260709 12:11:39.503448 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:39.502100 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.505308 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:39.520282 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.538265 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:39.543785 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60941.4us
I20260709 12:11:39.545603 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.547363 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.548285 10507 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:39.549379 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 68807.3us
I20260709 12:11:39.553279 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.556571 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.494670 14994 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:39.491092 15093 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:39.571878 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:39.575289 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79889.9us
I20260709 12:11:39.579141 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.581919 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.582350 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.583077 10506 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:39.583201 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:39.477528 15205 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.548774 15103 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.611526 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.518558 15131 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:11:39.574695 15199 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:11:39.574962 14994 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:39.608958 15063 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:39.591403 15061 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:39.581977 15232 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:39.621742 15103 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:39.631927 15160 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.004s
I20260709 12:11:39.632398 15160 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.634728 15093 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:39.625690 15178 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.635233 15093 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:39.636183 15093 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.591908 15243 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:11:39.624045 14994 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.637140 15178 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.624866 15238 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.630738 15207 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.642354 15238 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:39.636592 15103 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.595741 15230 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.628782 14972 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.646582 14972 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.647502 15160 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.649154 14901 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.581780 15196 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:11:39.651247 15213 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.628691 15205 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:39.656047 14994 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.658869 14994 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:39.712965 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.719647 10432 raft_consensus.cc:2410] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:11:39.723958 10433 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:11:39.724357 10510 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:39.727231 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:39.727850 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 144538us
W20260709 12:11:39.728013 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 136925us
I20260709 12:11:39.732558 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2604) took 1117 ms. Trace:
I20260709 12:11:39.730921 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.733234 10432 rpcz_store.cc:276] 0709 12:11:38.615078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.615235 (+   157us) service_pool.cc:224] Handling call
0709 12:11:39.732112 (+1116877us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400130454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:39.732369 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404887808}
I20260709 12:11:39.735324 10579 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:39.622206 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.739570 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:39.751783 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.752543 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.751682 10582 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:39.756902 10587 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:39.762550 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 254549us
I20260709 12:11:39.765144 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:39.766727 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 263225us
I20260709 12:11:39.716610 10331 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 9, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 9 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:39.781680 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:39.785645 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:39.779148 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48869.3us
I20260709 12:11:39.787827 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.788785 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:39.793670 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:39.794378 10323 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e4cd5d330f694dacb199bd61266c8f5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 52bae2d2caf94d97b1049ecd4f82a2df is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:39.797652 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.798054 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.799507 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:39.806861 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.810540 15251 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:39.823571 14868 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:39.824285 14868 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.824818 14868 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.827556 14997 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:11:39.830637 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 100082us
I20260709 12:11:39.830129 15053 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:11:39.812768 15251 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.836567 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.838229 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:39.840090 14559 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.840787 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:39.841387 14559 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.843263 14914 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:39.844085 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:39.843467 10331 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:39.844276 14914 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.842082 11217 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:39.843562 10580 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:39.849838 15251 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:39.857066 15186 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:39.863401 14969 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:39.864198 14969 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:39.865959 14969 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.869098 14559 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:39.872810 14914 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:39.880168 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46674.7us
I20260709 12:11:39.880905 14515 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.887603 15037 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:39.887519 15264 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:39.888103 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:39.888404 15037 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:39.887606 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.891541 15228 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:39.892153 15228 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:39.892628 15228 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:39.893488 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.881995 14515 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.894709 15138 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:39.895428 15245 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.896181 15138 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:39.896891 14515 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:39.898551 15138 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:39.900012 15138 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:39.907030 15138 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.890193 15037 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.903195 15199 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.909696 15199 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:39.912941 15199 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:39.894804 15178 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.890228 15264 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:39.915325 15264 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:39.900249 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:39.909874 15138 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:39.916882 15037 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:39.917264 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 56098.9us
I20260709 12:11:39.919875 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.924335 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:39.925622 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.851840 10331 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:39.929059 10508 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:39.924841 10504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:39.930105 10504 raft_consensus.cc:606] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:39.931619 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.939457 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:39.940917 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:39.954064 11217 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:39.956077 15093 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:39.957610 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 304754us
I20260709 12:11:39.962328 14914 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.971697 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:39.983029 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83039us
I20260709 12:11:39.983271 10507 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:39.985854 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:39.987551 10583 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:39.996603 15264 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:39.997380 15213 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:40.000578 15213 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:40.002671 15232 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:40.004159 15178 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:39.960640 15199 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:11:40.004638 15178 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:40.006049 15053 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:40.006402 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:40.007273 15196 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:40.006561 15053 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:40.005856 15178 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.008756 15196 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:40.014389 15053 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.015254 15196 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:40.015365 14566 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:40.010877 15230 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:40.016758 15053 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:40.016922 15230 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.016628 15196 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.017403 15230 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:40.018184 15196 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.011117 15232 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:40.014696 15178 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.019173 10504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:40.019953 15232 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:40.020495 15232 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:40.021680 15232 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.024067 10331 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 8 to 9, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 9 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.026582 10331 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.019944 10504 raft_consensus.cc:606] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:40.036530 10504 raft_consensus.cc:614] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.040319 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.041508 10586 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:40.044445 15251 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.046566 14959 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.047389 14959 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.050284 14959 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:40.050558 10511 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:39.998525 15264 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:40.056381 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47551.8us
I20260709 12:11:40.028873 10331 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 12, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.059576 10508 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:40.059257 15199 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:11:40.051124 15232 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.065846 15178 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:11:40.070905 15245 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:40.071501 15245 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:40.072624 15245 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.074646 15245 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.078644 14994 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.079607 14994 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.084712 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.086159 15063 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:40.086907 14994 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:40.066586 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.092551 10504 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
W20260709 12:11:40.102222 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43750us
I20260709 12:11:40.061794 15265 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:40.058147 14914 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.053874 14868 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:40.109683 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 49198.6us
I20260709 12:11:40.108286 15251 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:40.114903 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.115088 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:40.120843 14914 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:40.126441 11215 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:40.128221 15213 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:40.128479 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:40.128654 15213 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:40.141258 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.143751 15256 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:40.142323 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:40.130628 15213 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.149066 15256 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:40.146078 10507 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:40.134230 15245 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:11:40.154470 10507 raft_consensus.cc:614] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.161708 14972 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:40.144760 15228 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.162830 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:40.160832 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:40.167953 15245 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:11:40.168395 14868 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:40.154717 15256 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.169054 15265 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:40.169598 15265 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:40.174055 15213 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.170692 15265 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.175876 10586 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:11:40.176280 15265 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:11:40.179339 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 73280us
I20260709 12:11:40.180691 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.180792 15228 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:40.181659 14972 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:40.182181 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:40.182406 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.147770 10331 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 11, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.186033 14515 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:40.186712 14515 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:40.188007 14515 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.191954 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:40.192749 10507 raft_consensus.cc:606] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:40.193859 10504 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:11:40.195115 10507 raft_consensus.cc:614] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.196002 14972 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:40.199530 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.200454 10508 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:40.205294 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:40.205807 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.206780 15230 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.207337 14566 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:11:40.213140 15269 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:40.213285 10504 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:11:40.213593 15230 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:40.213619 15269 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:40.205775 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:40.207257 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.216153 11217 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:11:40.222196 10504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:40.229652 10504 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:40.223626 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:40.231578 10506 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:40.228202 10512 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:40.222277 10331 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 12, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.232865 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:40.233116 15269 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.234191 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.233492 10331 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 12, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 12 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.234802 15269 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:40.231643 15268 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:11:40.232264 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.230443 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.227246 14901 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:40.228406 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.236038 10504 raft_consensus.cc:614] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.234910 10512 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:40.240130 10507 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:40.242755 10513 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:40.243137 10512 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:40.242890 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:40.241235 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.245712 10505 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:11:40.247968 15053 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:40.248159 10331 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 14, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 14 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.248819 15053 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.271853 10585 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:40.250751 14810 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:11:40.252152 10509 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:40.281584 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:40.282922 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.283767 10509 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:40.284431 10435 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:11:40.284584 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.285406 10509 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:40.286360 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.241221 14901 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.244714 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:40.253120 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.255641 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.254215 15093 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:40.261746 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.262066 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.264057 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.267433 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.270429 10506 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:40.287127 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:40.302248 11215 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:40.287269 14901 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:40.303498 15196 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:11:40.288524 10507 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 11: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:40.308554 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.309303 10507 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:40.310202 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.286949 10509 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:40.249743 10510 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:40.291903  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:40.312449 10512 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:40.273221 15053 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:40.291546 14972 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:40.294659 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.293571 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.327847 10508 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:40.287969 10435 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 12:11:40.299716 10513 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
W20260709 12:11:40.329185 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.300634 10504 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:40.292443 15093 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.331691 10505 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:40.333360 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:40.299228 15265 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:40.339763 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:40.340842 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:40.337994  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:40.297472 14810 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:11:40.344429  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:40.344695 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.345697 10585 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:40.344627 10331 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 8 to 10, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 10 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.346696 15093 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:40.347702 10334 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 223) took 27503 ms (client timeout 15000 ms). Trace:
I20260709 12:11:40.349046 15138 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:40.349707 15138 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:40.299767 15228 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:11:40.349781 10579 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:11:40.350924 15138 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.346928 10331 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.339170 15265 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.366572 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:40.372476 15230 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:40.372949 15230 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:40.373327 15230 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:40.372278 10331 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 3 to 6, term changed from 7 to 10, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 10 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.369665 15265 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:40.386210 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.361027 15053 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:40.409211 10512 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:11:40.374799 10331 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.424149 10435 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:40.431926 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.417166 15053 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:40.435752 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.439556 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:40.436182 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:40.427742 14515 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:11:40.439100 15228 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:40.452605 15228 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:40.454217 15228 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:40.465109 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:40.461586 14972 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.450723 15227 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:40.422725 10331 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 2 to 3, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.469625 14972 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.472807 15227 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:40.489825 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:40.348161 10334 rpcz_store.cc:269] 0709 12:11:12.844171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.844387 (+   216us) service_pool.cc:224] Handling call
0709 12:11:40.347379 (+27502992us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:35.342864 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:35.354141 (+ 11277us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:35.354213 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:11:35.354242 (+    29us) tablet.cc:662] Decoding operations
0709 12:11:35.356610 (+  2368us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:35.356759 (+   149us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:35.356801 (+    42us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:11:35.482159 (+125358us) tablet.cc:701] Row locks acquired
0709 12:11:35.482213 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:11:35.482421 (+   208us) write_op.cc:270] Start()
0709 12:11:35.482601 (+   180us) write_op.cc:276] Timestamp: P: 1783599095482346 usec, L: 0
0709 12:11:35.482619 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:11:35.483555 (+   936us) log.cc:844] Serialized 7837 byte log entry
0709 12:11:36.775979 (+1292424us) op_driver.cc:464] REPLICATION: finished
0709 12:11:37.124150 (+348171us) write_op.cc:301] APPLY: starting
0709 12:11:37.124299 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:37.497569 (+373270us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:37.497733 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:37.728265 (+230532us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:37.728744 (+   479us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 12:11:37.728825 (+    81us) write_op.cc:312] APPLY: finished
0709 12:11:37.729445 (+   620us) log.cc:844] Serialized 180 byte log entry
0709 12:11:38.001675 (+272230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:38.030446 (+ 28771us) write_op.cc:454] Released schema lock
0709 12:11:38.034719 (+  4273us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:38.164712 (+129993us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:38.170606 (+  5894us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":110854,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":344016,"apply.run_cpu_time_us":46557,"apply.run_wall_time_us":941208,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87767,"num_ops":20,"prepare.queue_time_us":393723,"prepare.run_cpu_time_us":30947,"prepare.run_wall_time_us":497851,"raft.queue_time_us":500169,"raft.run_cpu_time_us":112952,"raft.run_wall_time_us":317193,"replication_time_us":1255590,"thread_start_us":1119061,"threads_started":5,"wal-append.queue_time_us":823304,"wal-append.run_cpu_time_us":42116,"wal-append.run_wall_time_us":2603930}]]}
I20260709 12:11:40.484601 15227 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:40.511958 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:40.503199 14959 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:40.509577 15265 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:40.536484 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.541072 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:40.501121 14901 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.528407 10331 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 7 to 8, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 8 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.498088 15053 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.527443 14959 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:40.559818 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:40.573096 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.557938 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:40.559789 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:40.586035 10434 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:40.587903 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117604us
I20260709 12:11:40.554790 14972 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:40.590157 10511 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:40.621042 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:40.617394 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.626291 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:40.627558 10585 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:40.582471 14959 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.630087 10331 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 6 to 9, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 9 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.632151 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.633198 10436 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:40.641430 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57645.8us
I20260709 12:11:40.647557 10331 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 12, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 12 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:40.658720 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:40.659766 10434 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:40.661417 15278 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:11:40.665764 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.667317 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:40.669838 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:40.670571 10583 raft_consensus.cc:2368] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
W20260709 12:11:40.671454 15276 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:11:40.671600 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.674937 14566 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:40.676591 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.676769 14566 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:40.706130 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:40.715600 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.685683 15269 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.725046 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:40.705621 15213 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:40.734395 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.738263 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.681270 15245 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:11:40.746526 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:40.724406 15037 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:11:40.748685 11178 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:40.735116 15213 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:40.762146 10580 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:11:40.787866 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:40.778450 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 115136us
I20260709 12:11:40.767573 15093 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:40.758810 15196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:38164 (request call id 11) took 2353 ms. Trace:
I20260709 12:11:40.805416 10583 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:40.806367 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:40.778622 14969 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:40.812569 10512 raft_consensus.cc:2368] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:40.813062 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:40.810091 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:40.815855 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.794301 14972 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:40.769503 15228 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.818084 10511 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:40.822492 10434 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:40.817639 10507 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:40.825337 10479 consensus_peers.cc:597] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfb1df1b6ba3414899a137b410511b26. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:11:40.829704 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:40.830770 10586 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:40.831136 15228 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:40.833065 14914 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:40.833340 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.833802 14914 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.836098 15288 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:40.836410 14914 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:40.839484 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:40.845093 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 166893us
I20260709 12:11:40.854158 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:40.836397 15256 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:40.824736 15280 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:11:40.872448 10508 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:11:40.815987 15251 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.884155 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:40.884459 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 176142us
I20260709 12:11:40.885035 10507 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:40.879204 14566 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.891592 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.891613 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:40.844071 15063 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:40.901227 15063 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.888115 14972 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.904001 15063 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:40.905166 14972 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:40.909029 15213 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.910578 15213 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:40.879771 15053 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:40.876124 15256 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:40.930919 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 91381.3us
I20260709 12:11:40.880178 15093 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.890872 15138 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:40.934473 15138 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.935530 15295 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:40.936169 15295 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:40.936932 15138 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:40.927404 15251 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.929726 15256 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.939778 15251 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:40.940614 15227 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.941879 10579 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:40.905203 15293 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:40.803462 15196 rpcz_store.cc:276] 0709 12:11:38.379343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.395187 (+ 15844us) service_pool.cc:224] Handling call
0709 12:11:38.560064 (+164877us) log.cc:844] Serialized 492 byte log entry
0709 12:11:39.308804 (+748740us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b54001002a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:11:39.309213 (+   409us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b54001002a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:11:40.725886 (+1416673us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39140,"raft.queue_time_us":48487,"raft.run_cpu_time_us":7649,"raft.run_wall_time_us":1069540,"spinlock_wait_cycles":36169216,"thread_start_us":162171,"threads_started":1,"wal-append.queue_time_us":188640,"wal-append.run_cpu_time_us":3402,"wal-append.run_wall_time_us":1175586}
I20260709 12:11:40.943252 15293 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:40.944554 15293 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:40.946183 15293 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:40.947893 15283 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:11:40.929704 15228 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.949237 15228 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:40.955756 15302 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:11:40.956367 15093 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:40.957543 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 70838.1us
I20260709 12:11:40.927923 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.980079 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:40.987639 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66885.8us
I20260709 12:11:40.988795 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:40.961177 15063 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:41.005348 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.005388 15228 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:41.009294 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 140961us
W20260709 12:11:41.010025 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 77681.6us
W20260709 12:11:41.016347 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.031787 15295 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:41.032697 15230 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.008s
I20260709 12:11:41.033192 15230 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.035233 10511 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:41.035599 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:40.991499 15213 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:41.037935 15213 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:41.044071 14972 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.044955 14972 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.045559 14972 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:41.070549 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52713.3us
W20260709 12:11:41.075412 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70194.8us
I20260709 12:11:41.075906 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.076598 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.078419 15227 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.003887 15265 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.079279 15302 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:11:41.081449 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 464620us
I20260709 12:11:41.090062 15299 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:41.092067 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.094447 15063 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.095072 15063 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.096623 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:41.097427 15063 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.097846 15288 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:41.098415 15288 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:41.099524 15288 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.100976 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.090871 15299 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.079612 14969 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:41.105581 15093 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:41.107474 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.114143 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.088739 15251 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:41.119604 14959 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.120472 15299 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.122386 14959 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.122450 15305 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.123128 11220 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:41.120042 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:41.124022 10438 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:41.128985 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.136150 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.136449 15196 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.139833 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.137248 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.136013 15305 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:41.140254 15196 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:41.145198 15305 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:41.148017 15305 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.140468 15178 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:11:41.156997 14972 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:41.147102 14994 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:41.157362 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:41.158027 15213 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:41.158612 15213 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:41.159026 15299 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.159106 15213 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:41.160789 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.161772 15302 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:41.162238 15302 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:41.162670 15302 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.134541 15230 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.176450 15297 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:11:41.183568 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43820.8us
I20260709 12:11:41.201812 15307 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:41.202793 15307 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.204123 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.205247 15293 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:41.205751 15293 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:41.205862 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.206048 15307 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.206293 15293 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.207974 15269 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:41.208746 15269 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:11:41.200641 15256 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.234066 10433 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:41.248703 15256 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.252810 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.226902 15299 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:41.260147 15256 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.260867 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.262023 15256 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:41.262367 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.264353 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.265182 14994 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:41.266059 14994 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.265874 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.268121 14994 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.268658 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.269524 14994 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:41.270002 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:41.287526 15302 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.290556 15230 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.296344 15230 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.301034 15230 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:41.291347 15302 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.302441 14901 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.303875 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.304579 10431 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:41.305747 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:41.306522 15230 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:41.291812 15299 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.294575 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:41.328157 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:41.329259 15299 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:41.329787 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:41.329980 15299 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.315979 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:41.330564 15251 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:41.319859 10511 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:41.322546 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:41.323567 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.332747 10403 leader_election.cc:400] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:41.333042 10403 leader_election.cc:403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:11:41.333443 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:41.333945 14972 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.334153 11219 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:41.334715 14972 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:41.324774 15213 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.336371 15178 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:41.273835 14994 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.276706 15288 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.337628 15178 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:41.338057 15178 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:41.337863 15288 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.338635 15213 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.340981 15288 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.343029 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.343835 10433 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:41.347666 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:41.337641 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:41.355580 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:41.357272 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.360136 14994 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.348955 15178 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:41.362149 15178 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:41.364322 15232 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:41.364861 15232 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.365335 15232 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.373222 14994 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:41.373761 14994 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.374173 14994 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:41.378194 14566 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.379129 11220 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:41.379842 15293 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.307217 15230 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.391227 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.397958 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.398934 10508 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:41.407783 10504 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:41.418298 14901 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.420111 14901 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.424299 14901 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.437294 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:41.441699 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 159006us
I20260709 12:11:41.391682 15230 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.446630 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:41.452543 10580 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:41.456782 11212 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:41.434569 15245 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.439867 15265 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:41.459650 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 92963.7us
W20260709 12:11:41.461053 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 255293us
I20260709 12:11:41.406522 15053 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:11:41.440224 15289 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:11:41.477089 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.467916 15037 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.468078 15305 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:41.471938 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.472635 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.480765 10583 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:41.481688 10432 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:41.475865 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:41.477957 10433 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.478363 10586 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:41.485538 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:41.486209 10580 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:41.486815 15305 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.487911 11219 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:41.489116 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.489738 10429 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:41.491386 11220 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:11:41.491529 15251 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:41.491782 11220 pending_rounds.cc:85] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 10
I20260709 12:11:41.491945 15251 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.481127 10431 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:41.492447 15251 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.493670 14972 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:41.494828 14972 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:41.492074 11220 pending_rounds.cc:107] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:11:41.495738 11220 raft_consensus.cc:2889] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:41.459311 15196 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.496397 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:41.505657 11212 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:41.495216 14972 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:41.512836 10580 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
W20260709 12:11:41.517163 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51010.4us
I20260709 12:11:41.497361 15293 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.519578 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:41.523375 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.520716 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:41.530097 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:41.530560 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:41.530905 10506 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.465026 15265 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.530298 15037 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.533269 15265 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.463574 15245 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.534552 15037 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:41.535575 15245 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.538025 11217 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:41.542845 10431 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 6.
I20260709 12:11:41.548368 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.509255 14810 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:41.552554 10433 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:41.519420 15309 ts_tablet_manager.cc:933] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.559187 10579 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:11:41.560433 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.561129 10432 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:41.561982 10429 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:41.564568 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.563755 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.509799 15196 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.543902 15256 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.576642 10506 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:41.565547 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.569198 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.580674 10435 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:41.581777 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:41.542770 14515 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:11:41.605793 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.607563 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:41.608489 11212 raft_consensus.cc:2410] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:11:41.609726 15213 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.612943 15213 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.615453 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.554477 15299 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:11:41.627678 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.610754 15293 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:41.559551 14810 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.583092 15256 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.654103 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.622237 15232 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.662622 10583 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:41.662743 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.665691 10506 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
W20260709 12:11:41.668380 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 64139.7us
I20260709 12:11:41.671891 10431 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:41.650183 15293 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:41.606071 14972 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.702062 15324 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:41.703667 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.704084 15322 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:11:41.704689 10438 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:41.709111 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.710655 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.712090 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.713188 10430 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:41.720388 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.669775 15307 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:11:41.685128 15213 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.724586 10438 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:41.728284 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:41.731119 15223 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.731850 15223 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.732569 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:41.733412 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.733475 10429 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:11:41.731307 10437 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:41.703698 15256 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.749994 10435 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:41.748522 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.752105 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.747141 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.752213 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:41.753471 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.763247 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.734215 15223 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:41.804935 10580 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:41.801815 10437 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:11:41.751255 14972 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.835713 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.825199 10504 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:11:41.837014 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.752727 15308 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:41.836690 10438 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:41.839885 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:41.805537 15317 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
W20260709 12:11:41.841861 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 75208.6us
I20260709 12:11:41.843158 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.846639 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.847325 10438 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:41.848114 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.848749 10506 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:41.849593 15196 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.851675 14994 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.852473 15232 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.854092 14994 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:41.855141 15309 tablet_copy_client.cc:323] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:11:41.856206 15303 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:41.856657 15303 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:41.860229 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.861353 15293 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.840556 10586 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:41.843545 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.866315 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:41.771442 15290 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:41.805517 15296 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:41.867749 10552 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:41.887197 10430 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:41.842873 15308 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:41.837951 15326 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:41.891005 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.891839 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.899051 10437 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:41.892366 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.893386 10435 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:41.900550 10429 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:41.886205 15290 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.890182 15305 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:41.911600 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.914412 10436 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.917308 15328 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:11:41.918205 15327 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:11:41.892413 14959 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:11:41.897897 15232 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:11:41.921407 15232 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:41.921770 15232 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:41.923269 10438 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:41.913380 10323 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e4cd5d330f694dacb199bd61266c8f5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 52bae2d2caf94d97b1049ecd4f82a2df is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:41.899489 15288 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.898715 15308 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.909415 15297 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:41.927563 15290 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:41.927919 15305 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.947785 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.893935 15296 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.922693 15232 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.952047 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:41.953341 10522 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet cfb1df1b6ba3414899a137b410511b26 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:11:41.926759 15326 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.955299 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.945284 15305 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:41.928789 15328 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:11:41.941283 15297 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.963721 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.963802 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.965606 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.967543 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:41.967873 15296 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:41.938421 15288 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:41.991273 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:41.951506 10478 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:41.976511 10504 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:41.958873 15230 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:41.981730 10430 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:41.981015 10433 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:41.987207 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.968250 10505 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:41.979646 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:41.996512 15339 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:11:41.996236 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.997485 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.998477 10585 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:41.999533 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.001757 11217 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:42.004184 10512 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:42.003043 10403 leader_election.cc:400] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:42.012121 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.012027 15330 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:42.014039 15330 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:42.014416 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.015807 15299 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:42.024236 10436 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:42.036048 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.982575 15297 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:42.038236 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:41.989216 15232 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.929875 15303 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.047425 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:42.044948 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.049003 10586 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:42.045590 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:41.975461 10522 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet cfb1df1b6ba3414899a137b410511b26 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-cfb1df1b6ba3414899a137b410511b26
W20260709 12:11:42.061448 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 109523us
I20260709 12:11:42.022794 14810 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:42.072906 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.073850 10583 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:42.075946 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:42.076200 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 167452us
I20260709 12:11:42.076767 10585 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:42.078112 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.078786 10511 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:42.079315 11212 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:42.079854 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.080132 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.082201 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:42.083416 10509 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:11:42.084969 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:42.086153 10403 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:42.088982 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.089299 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.090222 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:41.979413 15338 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
W20260709 12:11:42.095409 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 143995us
I20260709 12:11:42.095968 10505 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:42.096268 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.097021 10504 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:42.100659 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.097378 10510 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:42.096860 15332 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:11:42.103384 10509 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:42.074374 15299 raft_consensus.cc:2764] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:11:42.084169 15196 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:42.103132 15328 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:42.084054 15330 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:11:42.107123 15303 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:42.118208 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.114338 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.137128 10505 raft_consensus.cc:2410] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:11:42.113588 15196 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.144162 10585 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:11:42.146083 15245 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:42.147138 15347 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:42.148967 15347 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:42.149811 15328 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:42.151024 15328 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.144014 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:42.157521 10512 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:11:42.172927 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60319.1us
I20260709 12:11:42.155247 15196 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:42.155022 15230 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:42.200569 15343 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:11:42.212531 15288 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:42.215170 15315 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.216038 15315 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.219383 10522 tablet_copy_source_session.cc:215] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:11:42.220664 15351 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.221257 15351 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.223135  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:42.224479 15251 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.005s
I20260709 12:11:42.225111 15251 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.243966 10512 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
W20260709 12:11:42.269196 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 157711us
I20260709 12:11:42.270457 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:42.274642 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 190108us
W20260709 12:11:42.285161 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 187422us
I20260709 12:11:42.289701 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.290213 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:42.290592 10512 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
W20260709 12:11:42.290928 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 200531us
I20260709 12:11:42.292095 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:42.294432 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:42.292021 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:42.297360 10434 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:42.272665 15293 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:42.284385 15315 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:42.313001 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:42.322490 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.329481 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:42.303697 15288 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:42.334414 15357 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:42.349813 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.350864 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:42.353113 11217 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:42.295275 15299 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:11:42.374012 10432 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:42.334964 15357 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.376348 15245 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:42.356748 15309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfb1df1b6ba3414899a137b410511b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:42.345854 15288 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.399482 15356 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:42.410820 15245 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.415508 15251 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:42.351258 15364 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:42.376746 15293 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:42.427400 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 93874.7us
I20260709 12:11:42.347398 14972 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:11:42.428908 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:42.430039 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.431679 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:42.408120 15357 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:42.424111 15288 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:42.441022 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:42.445921 11217 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:42.434597 15326 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:42.432161 15351 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:42.429294 15297 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.486450 15265 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:42.487027 15265 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:42.488078 15265 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.489187 15265 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:42.490900 15326 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:42.491740 15326 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.489131 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:42.493180 10511 raft_consensus.cc:606] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:42.493563 10511 raft_consensus.cc:614] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:42.494081 15326 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:42.495895 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.496856 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:42.497521 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:42.497846 10581 raft_consensus.cc:2368] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:42.498172 10580 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:42.502053 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:42.502718 15367 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:42.503376 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.503652 15367 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:42.504063 10431 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:42.504469 15367 consensus_queue.cc:1243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:11:42.505280 15229 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.505448 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:42.505831 15229 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.506698 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:42.497715 15297 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:42.471596 15290 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.007s
I20260709 12:11:42.519251 10513 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:42.509554 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.536248 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.537348 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.539531 10585 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:42.551074 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:42.552587 10583 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:42.553834 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.505712 15351 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:42.554522 10583 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:42.473649 15356 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:42.554927 15351 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.554374 15366 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:42.556727 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.558781 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.562040 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:42.562716 10585 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:42.564173 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:42.565266 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46986.9us
I20260709 12:11:42.566195 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:42.567766 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:42.572559 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:42.573434 11217 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:42.585387 10583 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:42.571759 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:42.535523 15324 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:42.589121 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.556517 15332 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:42.595046 15138 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:42.596012 15371 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:42.596930 15371 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:42.604043 15053 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:42.604648 15053 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.604862 15229 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:42.557298 15356 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.605063 15053 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:42.591375 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:42.562186 15351 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:42.561569 15366 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.613399 15330 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:11:42.614290 15305 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.623s	user 0.013s	sys 0.000s
I20260709 12:11:42.577291 15232 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:42.581621 14566 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.579070 15256 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:42.579698 10580 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
W20260709 12:11:42.633769 15308 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.000s
I20260709 12:11:42.626487 10435 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
W20260709 12:11:42.636617 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45337.9us
I20260709 12:11:42.580575 15364 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:42.582104 15338 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:42.618206 15367 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.611429 15356 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:42.633553 15366 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:42.665999 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 101909us
I20260709 12:11:42.637591 15296 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:42.581830 15315 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:11:42.555507 15290 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.634042 15330 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:42.637713 15213 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:42.738076 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46492.2us
I20260709 12:11:42.763422 10552 leader_election.cc:400] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:42.763978 10552 leader_election.cc:403] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:42.676581 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:42.765658 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89206.8us
I20260709 12:11:42.768281 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:42.772859 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:42.671128 15364 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:42.656126 15305 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.719182 15338 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:42.632741 15324 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.785511 10433 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:42.790719 10582 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:42.665618 14566 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.744297 15296 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:42.795799 10433 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:42.781710 15303 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:11:42.810201 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:42.787088 15264 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:42.794603 15332 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:42.672340 15308 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.806882 15296 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:42.720088 15367 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.778235 15213 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:42.811435 15196 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 12:11:42.820055 10433 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.807740 15357 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.832767 10433 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:42.833031 14566 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:42.833781 14566 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:42.834065 15357 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:42.834388 14566 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:42.843302 15354 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 12:11:42.844026 15372 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:11:42.845789 15363 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:42.846130 15375 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:11:42.846596 15363 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:42.852981 15308 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:42.857013 15373 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:11:42.817756 15384 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:42.858325 15384 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.817778 15330 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.863270 15330 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:42.863945 15338 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:42.817984 15303 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:42.867137 15303 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.877588 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:42.878334 10512 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:42.822115 15377 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 12:11:42.822362 15319 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:11:42.882274 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.820322 15037 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:42.908319 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:42.909475 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:42.922650 10512 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:42.926927 14566 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:42.899173 15037 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:42.932876 15296 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.875379 15196 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.934931 10506 raft_consensus.cc:2368] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
W20260709 12:11:42.896190 15343 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:11:42.935763 15296 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:42.861545 15384 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:42.937920 10511 raft_consensus.cc:606] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:42.939056 10403 leader_election.cc:400] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:42.939566 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.946643 10403 leader_election.cc:403] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:42.946574 10511 raft_consensus.cc:614] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:42.936640 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:42.947371 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 148886us
I20260709 12:11:42.946914 10436 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:42.882527 15363 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:42.958317 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 159057us
W20260709 12:11:42.957717 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 169313us
I20260709 12:11:42.923740 15245 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.915226 15362 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.977247 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.978096 10436 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:42.962237 15299 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:42.987085 10433 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:42.988364 15351 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.994092 15326 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:42.999281 15376 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.000046 15376 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:43.001957 15358 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.651s	user 0.002s	sys 0.002s
I20260709 12:11:43.003629 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.005638 15387 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:43.006327 15387 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.006740 15387 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:43.007266 15384 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:42.960070 15232 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.007920 15384 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.008701 15232 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.005704 15371 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.010274 15384 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.010468 15371 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.012267 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:43.041342 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:42.934880 15288 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.065538 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:42.963892 15363 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.070309 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.070322 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:42.962023 15332 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:42.977320 15290 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:43.071110 15265 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.004s
I20260709 12:11:43.141842 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.144388 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:43.010914 15232 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.066010 15367 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.065279 15384 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:42.982189 15245 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:43.145562 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66840.5us
I20260709 12:11:43.157208 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.201778 10437 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.196076 10429 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:43.157729 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.201921 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:43.148046 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 163941us
I20260709 12:11:43.216127 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.217166 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2527) took 1223 ms. Trace:
I20260709 12:11:43.217521 10584 rpcz_store.cc:276] 0709 12:11:41.994238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.994557 (+   319us) service_pool.cc:224] Handling call
0709 12:11:43.215765 (+1221208us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b58001e3c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:11:43.217078 (+  1313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3394432}
I20260709 12:11:43.218205 15396 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:43.218859 15396 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:43.221295 15396 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:43.221917 15394 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:43.223569 15376 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.223851 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 519) took 1263 ms. Trace:
I20260709 12:11:43.224231 15376 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:43.225095 15376 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:42.987779 15362 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.224308 10432 rpcz_store.cc:276] 0709 12:11:41.959951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.960492 (+   541us) service_pool.cc:224] Handling call
0709 12:11:43.223668 (+1263176us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54003a98d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:43.223786 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844264320}
I20260709 12:11:43.228403 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:43.243352 10431 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:43.245831 10511 raft_consensus.cc:2368] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
W20260709 12:11:43.158849 15386 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:11:43.246863 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.245482 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:43.246446 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:43.198102 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 217392us
I20260709 12:11:43.142066 15303 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.155050 15382 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.247650 10581 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:43.244247 14972 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.231114 15305 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.207295 15367 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:43.249010 15368 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:11:43.143637 15315 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.244139 15265 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:43.317554 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72171.9us
I20260709 12:11:43.327162 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.222577 15394 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.202374 15332 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:43.246922 15351 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.325868 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.317274 15326 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:43.332042 15305 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.322337 14972 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.306998 15290 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:11:43.333658 10433 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:11:43.334573 15330 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:43.335822 15371 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.307669 15396 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:43.299751 15324 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:43.352365 15309 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.006s
I20260709 12:11:43.343103 11217 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
W20260709 12:11:43.331588 15053 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 12:11:43.334525 15245 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.123818 15388 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.257s
I20260709 12:11:43.348269 10437 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:43.226939 15384 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:43.350901 15364 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 12:11:43.354754 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.325984 15368 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.308674 15303 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.227316 15363 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.362494 15396 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:43.356958 15305 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.356258 15330 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.381946 15053 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.321357 15382 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:43.430580 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104805us
I20260709 12:11:43.479729 10504 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:43.344934 15265 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.341548 15367 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:43.445503 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 164939us
I20260709 12:11:43.464447 15407 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.461213 15396 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:43.470045 15305 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.510660 15384 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.529083 15382 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.531448 15265 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.532738 15332 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:43.533495 15332 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:43.534178 15332 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.340070 15315 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.431700 15371 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.433135 15376 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.543737 15371 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.551399 15196 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:43.557252 15299 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.000s
I20260709 12:11:43.558018 15299 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:43.546921 15371 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.559267 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2653) took 1232 ms. Trace:
I20260709 12:11:43.560869 15332 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.559670 10430 rpcz_store.cc:276] 0709 12:11:42.327049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:42.327191 (+   142us) service_pool.cc:224] Handling call
0709 12:11:42.347518 (+ 20327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:42.427895 (+ 80377us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:42.533633 (+105738us) log.cc:844] Serialized 14 byte log entry
0709 12:11:42.574079 (+ 40446us) log.cc:844] Serialized 14 byte log entry
0709 12:11:42.575422 (+  1343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:42.575493 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:42.575563 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:43.553969 (+978406us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b180030af60. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f
0709 12:11:43.555205 (+  1236us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b48000e0358. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f 00007f2c97af6f37 00007f2c97b0b55c 00007f2c97b0b508 00007f2c97b0b488 00007f2c97b0b443 00007f2c97b0a6ec 00007f2c96a34a96
0709 12:11:43.558941 (+  3736us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b540033c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:43.559008 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:43.559173 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":433142,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":4366,"raft.queue_time_us":1386,"raft.run_cpu_time_us":4182,"raft.run_wall_time_us":986797,"spinlock_wait_cycles":245270656,"thread_start_us":90238,"threads_started":1}
I20260709 12:11:43.568848 15296 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:43.569319 15296 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:43.576376 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.577219 11217 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:43.578578 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.579324 11212 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:43.581974 15213 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.592603 15315 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.617972 15399 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.618654 15399 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.621029 15399 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.486639 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.490118 15407 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.491885 15367 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.493301 15362 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:43.623618 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 137087us
I20260709 12:11:43.624010 15362 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:43.624445 15362 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.624812 15362 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.624958 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.625931 15407 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.570297 15296 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:43.627219 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.496973 15221 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.461522 15368 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.628214 15296 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:43.630030 15368 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.498900 15309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57904 (request call id 637) took 2379 ms. Trace:
I20260709 12:11:43.630636 15309 rpcz_store.cc:276] 0709 12:11:41.120659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.121004 (+   345us) service_pool.cc:224] Handling call
0709 12:11:41.564491 (+443487us) ts_tablet_manager.cc:934] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:11:42.283361 (+718870us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:11:42.838251 (+554890us) tablet_metadata.cc:665] Metadata flushed
0709 12:11:42.838435 (+   184us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:11:43.393738 (+555303us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:11:43.461621 (+ 67883us) tablet_replica.cc:206] Creating consensus instance
0709 12:11:43.498692 (+ 37071us) ts_tablet_manager.cc:989] Replica registered.
0709 12:11:43.498772 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":98890,"tablet-copy.queue_time_us":329058,"thread_start_us":187629,"threads_started":2}
I20260709 12:11:43.632500 15309 tablet_copy_client.cc:806] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:11:43.633114 15309 tablet_copy_client.cc:670] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:11:43.499526 15303 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.502118 15330 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:43.503667 15401 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:11:43.504372 15053 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.506145 15402 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:43.636354 15402 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:43.637373 15402 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.506421 15363 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.448424 15364 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.507813 15394 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:43.509727 15356 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.765s	user 0.004s	sys 0.008s
I20260709 12:11:43.640411 15356 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:43.642494 10435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54450 (request call id 517) took 1693 ms (client timeout 2042 ms). Trace:
I20260709 12:11:43.642781 15356 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:43.643177 10435 rpcz_store.cc:269] 0709 12:11:41.949063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.949227 (+   164us) service_pool.cc:224] Handling call
0709 12:11:43.642255 (+1693028us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54003d7cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:43.642407 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451034496}
I20260709 12:11:43.489372 15288 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.649329 15037 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:43.518734 15366 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.779s	user 0.007s	sys 0.006s
W20260709 12:11:43.651485 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 296849us
I20260709 12:11:43.651958 15366 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.652356 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.653105 10506 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:11:43.653298 15364 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.653905 15414 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.656165 15366 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:43.510881 15387 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.006s
I20260709 12:11:43.662266 15387 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:43.663692 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47370.2us
I20260709 12:11:43.664762 15387 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:43.519866 14972 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.665282 15326 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:43.650810 15037 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:43.671350 15406 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:43.672010 15406 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:43.673020 15406 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.677863 10579 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:11:43.679098 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54218.1us
I20260709 12:11:43.679477 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:43.680176 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.003s	sys 0.011s
W20260709 12:11:43.680524 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.003s	sys 0.011s
I20260709 12:11:43.681025 14972 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:43.681403 10578 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:43.681558 14972 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.683713 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:43.684329 14972 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.685120 14972 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.684331 10578 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.687819 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.688967 10581 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:43.689821 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:43.690476 10584 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:43.691588 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:43.692359 10579 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:43.695782 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.697688 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.698531 10586 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:43.702919 15366 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:11:43.703614 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49998.7us
I20260709 12:11:43.705387 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.706602 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.707351 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.709735 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.710646 11212 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:43.711253 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:43.658188 15362 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:43.716915 10506 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:11:43.733214 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.717995 10436 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:43.721257 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.734153 11219 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:43.723148 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.734892 10507 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:43.724920 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.726387 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.737481 10512 raft_consensus.cc:2368] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:43.740473 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.742375 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:43.742690 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:43.742763 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.743153 10504 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:43.744064 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.745474 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.729444 10581 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:43.729768 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.729857 10584 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:43.747334 10433 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:43.749188 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.752127 10504 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:11:43.719712 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.007s	sys 0.042s
W20260709 12:11:43.756065 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.008s	sys 0.042s
W20260709 12:11:43.756477 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 554641us
I20260709 12:11:43.729972 15364 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:43.768003 15364 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.770462 15364 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.730080 15256 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.685778 14972 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.741240 10508 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:43.770931 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:43.771968 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.798825 10437 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:11:43.798966 15346 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:43.799553 15346 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.799903 15346 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:43.799829 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:43.775664 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.776906 15362 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.803805 15232 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:11:43.804440 15232 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:43.777163 10432 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:43.805354 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.787881 15338 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.806797 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.806529 15338 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.807430 10432 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:43.809676 15338 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.788026 15399 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.787424 11220 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:43.810931 15399 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.811975 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:43.813277 15399 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.789278 15326 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:11:43.791280 15342 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.820986 15342 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.771603 15256 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.775336 10504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:43.823390 10504 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.823747 10504 raft_consensus.cc:614] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:43.776156 10584 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:43.824609 15256 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.775278 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.825624 10431 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:43.785566 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.827703 10438 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:43.825560 15342 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.793694 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.777335 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.829176 10435 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:43.829324 10430 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:43.786760 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.830245 11215 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:43.787070 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.803817 10434 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 8.
I20260709 12:11:43.793370 15397 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.832580 15397 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.779734 14972 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.792847 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:43.835668 10432 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:43.836404 15346 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.789093 15416 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.805346 15232 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.838331 15416 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:43.839968 10433 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:43.840595 15416 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:43.841207 15416 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:43.841531 15416 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.841972 15416 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:43.842545 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.842945 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.843282 11215 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:43.843781 11220 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:11:43.850661 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 122475us
I20260709 12:11:43.851555 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.650573 15396 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:43.866593 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.867849 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.868587 10506 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:43.868995 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.871042 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.872332 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.872448 10579 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:43.874676 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:43.875360 10506 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:43.873003 10513 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:43.885416 11217 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:43.888124 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.888859 10432 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:43.890893 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.893823 10432 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:43.893980 10437 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:43.890841 10403 leader_election.cc:400] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11.
I20260709 12:11:43.896070 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:43.897416 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.898409 10433 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
W20260709 12:11:43.853202 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 118322us
I20260709 12:11:43.901898 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.908097 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.908811 10431 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:43.909961 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:43.911795 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:43.912146 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 137678us
I20260709 12:11:43.912447 11217 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:43.913341 10552 leader_election.cc:400] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:43.913645 10552 leader_election.cc:403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:11:43.915942 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.916570 10436 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:43.927392 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:43.928244 10508 raft_consensus.cc:2368] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
W20260709 12:11:43.929164 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 77707.5us
I20260709 12:11:43.931283 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:43.931651 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.931752 10432 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:43.929905 10404 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.933240 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.931983 10430 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:43.943516 10580 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:11:43.834832 15397 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:43.944571 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.944710 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.943312 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.938284 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.939309 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.938604 10435 raft_consensus.cc:2376] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13. Re-sending same reply.
I20260709 12:11:43.946249 10403 leader_election.cc:400] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:43.942226 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.946153 11217 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:43.932674 10513 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:43.948627 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.939319 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:43.952603 10437 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.956256 10506 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:43.946614 10403 leader_election.cc:403] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:43.949231 10438 raft_consensus.cc:2376] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15. Re-sending same reply.
I20260709 12:11:43.934767 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:43.960471 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:43.959685 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.960575 10578 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:43.961975 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.952808 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:43.884110 15387 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.963089 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:43.968650 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:43.969215 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2737) took 1408 ms. Trace:
I20260709 12:11:43.971593 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:43.974853 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.909816 15364 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.912051 15330 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.995785 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.988438 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:43.996742 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.981709 11217 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:43.983451 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.941943 15395 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:43.940953 15303 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.020520 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.021322 10580 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.981140 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:44.025868 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 81363.5us
I20260709 12:11:44.026773 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:44.028043 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:44.028702 10506 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:43.981389 10585 rpcz_store.cc:276] 0709 12:11:42.543866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:42.565217 (+ 21351us) service_pool.cc:224] Handling call
0709 12:11:42.565513 (+   296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:42.583505 (+ 17992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:42.583542 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:42.584185 (+   643us) log.cc:844] Serialized 52 byte log entry
0709 12:11:43.015551 (+431366us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b48000b19d8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c97a08aa1 00007f2c979ffb37 00007f2c979ff9ca 00007f2c97af726f 00007f2c97af6f37 00007f2c97b0b55c 00007f2c97b0b508 00007f2c97b0b488 00007f2c97b0b443 00007f2c97b0a6ec 00007f2c96a34a96
0709 12:11:43.393978 (+378427us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b18003769c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:43.720181 (+326203us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:43.866227 (+146046us) log.cc:844] Serialized 14 byte log entry
0709 12:11:43.951378 (+ 85151us) log.cc:844] Serialized 14 byte log entry
0709 12:11:43.951617 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:43.951710 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:43.951761 (+    51us) raft_consensus.cc:1685] finished
0709 12:11:43.951774 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:43.951903 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201715,"prepare.queue_time_us":163183,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":103802,"raft.run_cpu_time_us":1631,"raft.run_wall_time_us":443597,"spinlock_wait_cycles":99299584,"thread_start_us":325942,"threads_started":1,"wal-append.queue_time_us":1015985}
I20260709 12:11:43.982185 10437 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:44.011411 15355 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:43.932264 15366 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:44.018265 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 55277.5us
W20260709 12:11:43.992823 15228 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.022s	sys 0.087s
I20260709 12:11:43.967165 15367 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:43.966931 15338 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:44.038447 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57382.5us
I20260709 12:11:43.967404 14959 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:43.962656 15407 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:44.042870 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.959551 15406 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:11:44.020857 15394 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:43.923454 15290 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:11:44.047277 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:43.940068 15399 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:43.962186 14901 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:43.982072 10582 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:43.975101 14994 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:43.973997 15402 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.974792 15302 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:44.057886 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:43.986105 14810 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:43.993765 15397 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.949075 14972 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:11:44.021368 15395 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.049031 15297 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:44.090634 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.190754 10336 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 8 to 12, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:44.198838 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.199767 10585 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 7: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:44.200700 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.176517 15402 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.201936 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.202695 10585 raft_consensus.cc:2368] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:44.203186 10584 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:44.055743 15355 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.171131 15399 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.206607 15355 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:43.962069 15390 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:44.053081 15342 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:44.212720 15342 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:44.213748 15342 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.082520 15138 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:44.021824 15330 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.219347 15330 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.208339 10336 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 8 to 11, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:44.220791 10336 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 12 to 13, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 13 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:44.016501 15288 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:43.974953 15409 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:44.085422 15367 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.128901 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.037175 15356 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:11:44.044031 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46310.2us
W20260709 12:11:44.088621 15228 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.022s	sys 0.090s
W20260709 12:11:44.159660 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 112449us
I20260709 12:11:43.995054 15387 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.166322 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.085940 15338 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.132782 15407 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.279407 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:44.280411 10507 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:44.020522 15364 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.170015 15302 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.153029 15346 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.197005 10331 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 110) took 9061 ms. Trace:
I20260709 12:11:44.085692 15410 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 12:11:44.113211 15396 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.169674 15303 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:11:44.177345 15334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.032s
I20260709 12:11:44.204650 15402 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:44.208261 14810 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:44.209964 15357 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:44.080307 15366 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:44.223260 15296 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.003s
I20260709 12:11:44.223943 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:44.306160 11217 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 3.
I20260709 12:11:44.224951 15399 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.229151 15391 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.234s
W20260709 12:11:44.229789 15411 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:11:44.312942 15391 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:44.233825 15395 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.313396 15391 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:44.236286 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:44.185436 15351 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:11:44.246193 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:44.300784 15302 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:44.260807 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.285316 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:44.247210 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.287319 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:44.167608 15394 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:44.286273 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:44.299324 15356 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.299062 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.260411 15367 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.333528 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.255622 14972 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.301571 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:44.256153 15288 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.337466 10587 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 11: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:44.303112 10336 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 4 to 10, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:44.281819 15338 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.277479 15409 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:44.343789 15409 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.336438 15356 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.314424 15391 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.346582 15346 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:44.347450 15391 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:44.347239 15346 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.349630 15346 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.350589 15387 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.352494 15394 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.310079 10579 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:44.310231 10404 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:44.355504 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 45382.4us
I20260709 12:11:44.287927 15407 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:44.358835 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.353933 15330 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:11:44.364393 15420 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 12:11:44.364612 15330 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:44.365708 15330 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.366235 15256 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:11:44.366740 15256 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.296432 10331 rpcz_store.cc:276] 0709 12:11:35.135263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.135500 (+   237us) service_pool.cc:224] Handling call
0709 12:11:44.196897 (+9061397us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:41.209375 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:41.210144 (+   769us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:41.210199 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:11:41.210211 (+    12us) tablet.cc:662] Decoding operations
0709 12:11:41.211785 (+  1574us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:41.211899 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:41.211946 (+    47us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:11:41.309345 (+ 97399us) tablet.cc:701] Row locks acquired
0709 12:11:41.309416 (+    71us) write_op.cc:260] PREPARE: finished
0709 12:11:41.309650 (+   234us) write_op.cc:270] Start()
0709 12:11:41.309856 (+   206us) write_op.cc:276] Timestamp: P: 1783599101309569 usec, L: 0
0709 12:11:41.309902 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:11:41.310814 (+   912us) log.cc:844] Serialized 6940 byte log entry
0709 12:11:41.932385 (+621571us) op_driver.cc:464] REPLICATION: finished
0709 12:11:42.201454 (+269069us) write_op.cc:301] APPLY: starting
0709 12:11:42.228175 (+ 26721us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:42.397505 (+169330us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:42.397676 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:42.872617 (+474941us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:42.873077 (+   460us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:11:42.873146 (+    69us) write_op.cc:312] APPLY: finished
0709 12:11:42.873690 (+   544us) log.cc:844] Serialized 164 byte log entry
0709 12:11:42.933008 (+ 59318us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adc32a 00007f2c92a61f14 00007f2c92a61ec3 00007f2c92a61e38 00007f2c92a61df3 00007f2c92a610dc 00007f2c92a8ae16 00007f2c92a72cbb 00007f2c92a5d39d 00007f2c92a60d75
0709 12:11:43.364573 (+431565us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:43.462165 (+ 97592us) write_op.cc:454] Released schema lock
0709 12:11:43.489595 (+ 27430us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:43.517775 (+ 28180us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:43.517807 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":381685,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":227548,"apply.run_cpu_time_us":44042,"apply.run_wall_time_us":1330902,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":234634,"prepare.run_cpu_time_us":45229,"prepare.run_wall_time_us":297030,"raft.queue_time_us":255234,"raft.run_cpu_time_us":27078,"raft.run_wall_time_us":1054026,"replication_time_us":587518,"spinlock_wait_cycles":6988544,"thread_start_us":748128,"threads_started":5,"wal-append.queue_time_us":698507,"wal-append.run_cpu_time_us":21346,"wal-append.run_wall_time_us":1203439}]]}
W20260709 12:11:44.310674 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 283994us
I20260709 12:11:44.315318 10511 raft_consensus.cc:606] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
W20260709 12:11:44.303119 15334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.012s	sys 0.034s
I20260709 12:11:44.311430 15357 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:44.317260 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71161.4us
I20260709 12:11:44.260639 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:44.384805  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:11:44.385551 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 125027us
I20260709 12:11:44.302868 15303 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.387261 15303 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:44.388279  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:44.389508  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:44.334362 10552 leader_election.cc:400] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:44.290166 10580 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:44.391163 10552 leader_election.cc:403] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:11:44.391769 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57631.5us
I20260709 12:11:44.336117 10581 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:44.332363 15368 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:44.394455 15368 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:44.395330 15223 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:44.394902 15368 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.404247 15397 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:44.405102 15397 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:44.357316 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:44.361333 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.368651 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:44.370666 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:44.421885 11177 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.545s (SENT) [suppressed 20 similar messages]
I20260709 12:11:44.406365 15397 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.439414 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:44.440814 15406 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:44.441394 15406 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:44.441874 15406 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.443475 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:44.447551 15326 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:44.448257 15326 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:44.449566 15326 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:11:44.456280 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.471024 15232 raft_consensus.cc:993] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:11:44.471839 15232 raft_consensus.cc:1081] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:11:44.310451 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.326589 15296 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.375795 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.376480 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:44.474373 15296 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:44.475107 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98740.7us
I20260709 12:11:44.377381 10512 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:44.476243 15223 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:11:44.317000 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.476871 15296 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:44.332692 15410 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:44.477809 15410 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.480187 15410 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.476384 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:44.481341 15396 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:11:44.477128 10583 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:44.481438 15368 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.482234 15396 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:44.484866 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:44.356449 15288 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:44.485675 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.358055 15399 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:44.346650 15356 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:44.378265 15037 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.002s
I20260709 12:11:44.358913 15346 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.490528 10437 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:44.508466 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.474285 10586 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.491333 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.492393 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.493723 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.365720 15364 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.375276 15330 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.376041 15256 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:44.495316 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.530553 10583 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.495816 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:44.496441 10580 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:11:44.533442 10435 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:44.488281 15410 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:44.497488 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:44.490229 14994 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.503134 15356 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.359573 14972 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.347276 14810 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.567384 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55454
I20260709 12:11:44.504830 15357 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.505972 15366 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:44.538589 15416 consensus_meta.cc:287] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.000s
I20260709 12:11:44.509102 10584 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:44.512568 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.510450 10582 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:44.526494 15364 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:44.526795 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.534657 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:44.587494 10429 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.535986 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:44.537691 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52908.6us
I20260709 12:11:44.539458 10438 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:44.498409 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.490026 15399 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.547082 10432 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:11:44.560299 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.564695 15356 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.565335 14972 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.569856 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:44.578696 10587 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:44.579384 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.579815 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 8 to 10, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 10 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:44.532989 15346 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.612075 10587 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:44.613793 10336 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:44.539232 15396 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.550583 15410 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.623579 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.569137 14994 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.625293 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.608453 15399 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.630586 10336 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 5 to 8, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:44.611832 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:11:44.636317 10581 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:44.636902 10581 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.642834 10429 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:44.602031 15362 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:11:44.652928 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.601845 15416 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.616252 15296 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:11:44.671451 15356 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.673336 15410 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.631165 15364 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.676397 10584 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:44.682257 15368 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.687832 15416 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.688832 10336 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 2 to 11, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:44.707703 10336 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 6 to 13, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:44.692474 14994 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.709614 15302 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.711702 15367 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.713491 15395 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.714515 15367 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.715890 15346 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.696827 15256 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.703651 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:44.727291 10583 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:44.721890 15288 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.729548 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:44.710201 10336 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 6 to 12, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:44.743398 10587 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:11:44.695421 15406 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.732524 10581 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.752992 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:44.753906 10579 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:44.755374 10513 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:44.756273 10336 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 9 to 11, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } }
I20260709 12:11:44.746188 10584 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:44.767611 15302 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.706372 15364 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:44.768877 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:44.751387 15037 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:11:44.810122 11178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.086s	sys 0.164s
W20260709 12:11:44.810598 11178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.086s	sys 0.164s
I20260709 12:11:44.751363 15256 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:44.758749 15342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.024s	sys 0.023s
I20260709 12:11:44.815550 11217 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:11:44.815421 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:44.820147 11220 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:44.822221 10506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.013s	sys 0.038s
W20260709 12:11:44.822656 10506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.013s	sys 0.038s
I20260709 12:11:44.828632 10336 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 10 to 11, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:44.835004 11217 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:44.816839 15342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.025s	sys 0.023s
W20260709 12:11:44.838527 10510 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.027s	sys 0.022s
W20260709 12:11:44.866029 10510 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.028s	sys 0.022s
W20260709 12:11:44.844515 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.809557 15351 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.841069 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:44.873143 10513 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:44.873581 10513 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:44.811476 15362 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:44.863616 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:44.836328 15315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.009s	sys 0.036s
I20260709 12:11:44.821867 15037 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:44.819193 15302 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.889237 10586 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:44.882345 11220 raft_consensus.cc:2410] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:11:44.888111 10581 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:44.876685 10336 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 6 to 7, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 7 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:44.874516 10513 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.871155 15395 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:44.880697 15303 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.902542 15303 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:44.887708 15315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.009s	sys 0.037s
W20260709 12:11:44.881243 15338 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.007s
I20260709 12:11:44.904592 15338 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.906589 10512 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:44.813992 15228 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:11:44.914803 15228 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:44.915217 15228 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:44.915695 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:44.916201 15228 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.917479 15228 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:44.881646 15362 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.918918 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:44.919443 15362 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:44.923295 10433 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.015s	sys 0.050s
W20260709 12:11:44.923741 10433 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.015s	sys 0.050s
I20260709 12:11:44.925393 11178 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:44.881398 15407 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 12:11:44.930109 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:44.931010 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:44.937784 10509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.013s	sys 0.049s
W20260709 12:11:44.966082 10578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.020s	sys 0.025s
W20260709 12:11:44.965987 10509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.013s	sys 0.049s
W20260709 12:11:44.966527 10578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.020s	sys 0.025s
I20260709 12:11:44.967854 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2384) took 1044 ms. Trace:
I20260709 12:11:44.971200 10336 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 7 to 10, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 10 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:44.872120 15366 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:11:44.979080 10508 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:44.928491 15338 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.009140 10336 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:44.887624 15351 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:44.965945 15315 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:45.009625 11177 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.002s	sys 0.020s
W20260709 12:11:45.017323 11177 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.003s	sys 0.020s
I20260709 12:11:45.018754 10336 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:11:45.019999 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 598135us
W20260709 12:11:44.982969 15397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.004s	sys 0.014s
W20260709 12:11:44.982421 15326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.007s	sys 0.010s
I20260709 12:11:44.924175 15302 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:45.025393 15397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.014s
I20260709 12:11:45.027513 15302 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:45.028244 15302 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:45.028752 15302 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:45.029310 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:45.029881 15302 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.031237 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:45.034598 15326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.010s
W20260709 12:11:45.051222 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.010s	sys 0.014s
W20260709 12:11:45.051622 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.010s	sys 0.014s
I20260709 12:11:44.943460 15407 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.022461 10336 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:11:45.064365 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.978324 15432 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:11:45.099699 10336 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 1 to 12, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 12 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:45.018131 15315 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:45.102106 15394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.004s	sys 0.026s
W20260709 12:11:45.106933 10402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.015s	sys 0.016s
W20260709 12:11:45.108078 10402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.015s	sys 0.016s
I20260709 12:11:45.109252 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.108834 10336 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:11:45.111099 15394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.004s	sys 0.027s
I20260709 12:11:45.112159 10507 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:11:45.113499 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.016888 15429 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 12:11:45.115799 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:45.110769 15303 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:45.117018 15338 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.117697 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:45.056353 15433 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:45.151914 10511 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:45.144951 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.155278 15433 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:45.155512 10512 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:45.178256 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2678) took 1271 ms. Trace:
I20260709 12:11:45.178825 10433 rpcz_store.cc:276] 0709 12:11:43.906904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:43.920416 (+ 13512us) service_pool.cc:224] Handling call
0709 12:11:43.957847 (+ 37431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:44.000962 (+ 43115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:44.001018 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:44.001723 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 12:11:44.214383 (+212660us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b18001ac6e0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:44.924157 (+709774us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:44.924218 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.924336 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:45.177790 (+253454us) raft_consensus.cc:1685] finished
0709 12:11:45.177821 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.178148 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14880,"raft.queue_time_us":11241,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":22967,"spinlock_wait_cycles":41631104,"thread_start_us":709544,"threads_started":1,"wal-append.queue_time_us":1047495}
I20260709 12:11:45.181391 15407 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.183759 15351 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:45.186931 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 69353.7us
I20260709 12:11:45.187249 15289 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:45.188695 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:45.190824 15396 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:45.191412 15396 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:45.192543 15396 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:44.968389 10509 rpcz_store.cc:276] 0709 12:11:43.923727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:43.923875 (+   148us) service_pool.cc:224] Handling call
0709 12:11:43.924218 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:43.924469 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:11:44.153075 (+228606us) log.cc:844] Serialized 14 byte log entry
0709 12:11:44.967098 (+814023us) log.cc:844] Serialized 14 byte log entry
0709 12:11:44.967358 (+   260us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:44.967419 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:44.967504 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.967601 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:44.967774 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2265,"prepare.queue_time_us":61058,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":13800,"raft.queue_time_us":1649,"thread_start_us":991617,"threads_started":2}
I20260709 12:11:45.107767 15288 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.049789 15037 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:45.194818 15399 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.001s
I20260709 12:11:45.105908 15315 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:45.226588 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:45.117662 15406 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.237632 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:45.156359 15433 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.239102 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:45.240337 15366 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:45.241878 15366 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:45.242246 15366 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.232002 15326 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:45.243078 15397 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:45.243534 15397 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.243916 15397 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:45.245162 15436 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:11:45.245718 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:45.243088 15326 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.246621 15342 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:45.246954 15326 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.242619 15366 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:45.154392 15303 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.255961 10513 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:45.256544 10437 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:45.254750 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.257802 10429 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:45.194464 15338 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.260614 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.261420 10429 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:45.265326 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.275965 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.276965 10433 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:45.277338 10432 raft_consensus.cc:2393] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 9: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:45.276110 15396 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:45.283064 15437 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:11:45.253333 15416 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:45.234615 15399 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.291244 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.291925 10438 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:45.293558 10429 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:45.253494 15391 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:11:45.300788 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.284669 15416 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.305055 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.306736 10431 raft_consensus.cc:2393] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 4: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:45.307662 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:45.308076 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.306730 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.312247 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.313210 10437 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:45.313310 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.315590 10434 raft_consensus.cc:2368] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14.
I20260709 12:11:45.288975 14959 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:45.316660 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.317955 10430 raft_consensus.cc:2368] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:45.306833 15416 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:45.323462 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.324122 10433 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:45.320922 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:45.290884 10569 thread.cc:641] tablet-delete [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.027s	sys 0.042s
W20260709 12:11:45.330647 10569 thread.cc:608] tablet-delete [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.027s	sys 0.042s
I20260709 12:11:45.335151 15399 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.336041 15399 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:45.339651 10437 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:45.339866 15416 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:45.304984 14810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.048s	sys 0.103s
W20260709 12:11:45.340993 14810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.048s	sys 0.103s
I20260709 12:11:45.342108 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.342152 15288 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:11:45.346946 15315 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.352294 15315 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.349886 15416 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:45.348609 10429 raft_consensus.cc:2376] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12. Re-sending same reply.
I20260709 12:11:45.288991 15394 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.303239 15256 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:45.387974 10582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.032s	sys 0.038s
W20260709 12:11:45.388414 10582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.032s	sys 0.038s
W20260709 12:11:45.391685 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.012s	sys 0.020s
W20260709 12:11:45.392418 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.013s	sys 0.021s
W20260709 12:11:45.394299 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.395391 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:45.396127 11220 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:45.396386 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:45.397459 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.398741 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.399665 10506 raft_consensus.cc:2410] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:45.398741 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:45.400293 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:45.401682 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:45.402442 11220 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.401041 11217 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:11:45.380923 15409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.014s	sys 0.051s
W20260709 12:11:45.407778 15409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.014s	sys 0.051s
I20260709 12:11:45.322533 15346 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:45.356890 14959 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:45.444523 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.450374 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.467497 10432 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
W20260709 12:11:45.468189 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.449895 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.450488 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.306928 15406 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:11:45.384558 15334 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:45.316143 15351 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.122s
W20260709 12:11:45.483508 10585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.020s	sys 0.058s
W20260709 12:11:45.484745 10585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.020s	sys 0.059s
I20260709 12:11:45.443117 15346 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.497079 14959 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.499207 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.500137 10429 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 11: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:45.474469 15409 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:45.470336 15445 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:45.394416 15256 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.477754 15334 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:45.437023 15394 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.504148 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:45.506925 10506 raft_consensus.cc:2368] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
W20260709 12:11:45.493840 11178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.021s
I20260709 12:11:45.505352 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:45.508287 11178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.000s	sys 0.022s
W20260709 12:11:45.509456 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 578501us
I20260709 12:11:45.444080 15407 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:11:45.513693 10510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.027s	sys 0.042s
W20260709 12:11:45.569698 10510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.028s	sys 0.043s
I20260709 12:11:45.500022 15346 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.520036 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.575477 10430 raft_consensus.cc:2368] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:45.521966 10429 raft_consensus.cc:1240] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:11:45.521507 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.576270 10437 raft_consensus.cc:2368] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:45.524478 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.531514 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:45.577551 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53182.3us
I20260709 12:11:45.533164 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.533782 11217 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:11:45.507154 15445 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.503041 15451 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:45.538640 11220 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:45.538728 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:45.586022 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:45.586670 10512 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:45.560245 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.026s	sys 0.029s
I20260709 12:11:45.598251 10513 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:45.599663 15449 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55454 (request call id 74) took 1049 ms. Trace:
I20260709 12:11:45.600147 15449 rpcz_store.cc:276] 0709 12:11:44.563189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.567019 (+  3830us) service_pool.cc:224] Handling call
0709 12:11:45.599089 (+1032070us) ts_tablet_manager.cc:1151] Deleting tablet 0b474d9b241d4a698702287ee1bbe3b2
0709 12:11:45.599184 (+    95us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:11:45.599520 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":996951,"thread_start_us":762672,"threads_started":1}
I20260709 12:11:45.605429 15357 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:45.606087 15357 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:45.607154 15357 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.608336 15256 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:45.609155 15362 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.000s
I20260709 12:11:45.612389 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:45.613674 15351 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.591133 15407 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.614384 15351 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.616206 15407 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:45.610903 15362 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.618660 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:45.619235 15451 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:45.620497 10509 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14.
I20260709 12:11:45.620287 15451 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.621354 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:45.567018 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:45.529273 15434 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:11:45.624366 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2565) took 1349 ms. Trace:
I20260709 12:11:45.625676 10510 rpcz_store.cc:276] 0709 12:11:44.274565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.274729 (+   164us) service_pool.cc:224] Handling call
0709 12:11:44.275093 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:44.275376 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:11:44.296404 (+ 21028us) log.cc:844] Serialized 14 byte log entry
0709 12:11:45.570513 (+1274109us) log.cc:844] Serialized 14 byte log entry
0709 12:11:45.623722 (+ 53209us) log.cc:844] Serialized 14 byte log entry
0709 12:11:45.623959 (+   237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:45.624009 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:45.624058 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.624129 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.624248 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40180,"prepare.queue_time_us":29880,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":60962,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":290,"thread_start_us":1273087,"threads_started":2,"wal-append.queue_time_us":801890}
W20260709 12:11:45.632580 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.026s	sys 0.029s
I20260709 12:11:45.479563 15397 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.621604 15451 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:45.647096 10438 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:45.649364 10404 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:45.649989 10438 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:45.653668 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:45.656867 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.657518 10438 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:45.583055 15443 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:11:45.659152 15446 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.670159 15447 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:11:45.670485 15452 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:11:45.670848 15447 raft_consensus.cc:697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:45.670590 15446 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.672278 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2689) took 1175 ms. Trace:
I20260709 12:11:45.671939 15447 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.672677 10436 rpcz_store.cc:276] 0709 12:11:44.497230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.497383 (+   153us) service_pool.cc:224] Handling call
0709 12:11:44.497726 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:44.510349 (+ 12623us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:44.570488 (+ 60139us) log.cc:844] Serialized 14 byte log entry
0709 12:11:44.686294 (+115806us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18001edd80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1
0709 12:11:44.686746 (+   452us) spinlock_profiling.cc:243] Waited 9.04 ms on lock 0x7b18001edd80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:44.689304 (+  2558us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18001edd80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f23a 00007f2c92ae9a2a
0709 12:11:45.037251 (+347947us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:45.037879 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 12:11:45.038054 (+   175us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:45.038101 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.038195 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:45.672040 (+633845us) raft_consensus.cc:1685] finished
0709 12:11:45.672067 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.672209 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12144,"prepare.queue_time_us":60083,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":128694,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":4770,"spinlock_wait_cycles":27671936,"thread_start_us":335758,"threads_started":1,"wal-append.queue_time_us":877549}
I20260709 12:11:45.667557 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:45.510258 15409 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.669137 10429 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:45.583963 15303 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.632614 15445 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:45.680941 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:45.663089 10512 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:45.689968 10429 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:45.660781 15399 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.705237 10513 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
W20260709 12:11:45.677667 15444 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.194s	user 0.000s	sys 0.003s
I20260709 12:11:45.679864 15446 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:45.717399 10438 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7.
W20260709 12:11:45.718056 10553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.036s	sys 0.094s
W20260709 12:11:45.717799 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.026s	sys 0.012s
W20260709 12:11:45.726467 10553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.036s	sys 0.094s
I20260709 12:11:45.697479 15303 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:45.729247 11220 raft_consensus.cc:1240] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 3. Current term is 4. Ops: []
I20260709 12:11:45.730932 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:45.726519 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.026s	sys 0.013s
I20260709 12:11:45.732599 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.605413 15366 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.742105 10511 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:45.744331 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:45.748278 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.749276 15407 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:45.748852 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.743127 14810 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:11:45.742836 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:45.745668 15256 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:45.751709 15351 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.752388 14810 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:45.752579 15352 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:45.753062 14810 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:45.753383 15448 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:11:45.754208 14810 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.754947 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 0b474d9b241d4a698702287ee1bbe3b2 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:11:45.755834 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.756047 15455 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.756896 14810 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:45.753185 15352 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.758384 15352 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.760694 15390 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:45.762085 15429 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:45.762452 15429 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:45.763449 15429 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.764923 15429 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:45.768967 15438 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.241s	user 0.002s	sys 0.000s
I20260709 12:11:45.751729 15407 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:45.757624 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.770767 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2767) took 1774 ms. Trace:
I20260709 12:11:45.771623 10578 rpcz_store.cc:276] 0709 12:11:43.996042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.022925 (+ 26883us) service_pool.cc:224] Handling call
0709 12:11:44.079929 (+ 57004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:44.090164 (+ 10235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:44.116325 (+ 26161us) log.cc:844] Serialized 14 byte log entry
0709 12:11:45.003754 (+887429us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:45.004505 (+   751us) log.cc:844] Serialized 26 byte log entry
0709 12:11:45.004693 (+   188us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:45.004740 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.004842 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:45.768342 (+763500us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:45.770370 (+  2028us) raft_consensus.cc:1685] finished
0709 12:11:45.770501 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.770684 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36363,"prepare.queue_time_us":72031,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3807,"raft.queue_time_us":70211,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":251,"spinlock_wait_cycles":17089280,"thread_start_us":850159,"threads_started":1,"wal-append.queue_time_us":1367019}
I20260709 12:11:45.773880 15338 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.774475 15338 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.775095 15407 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:45.775024 15338 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.777272 15338 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:45.778163 10403 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:45.783574 15407 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:45.784682 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2782) took 1057 ms. Trace:
I20260709 12:11:45.785034 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.786754 10403 leader_election.cc:400] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:45.779785 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.787695 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:45.788616 10508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.014s	sys 0.031s
W20260709 12:11:45.788993 10508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.014s	sys 0.031s
I20260709 12:11:45.756651 15455 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.794144 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:45.791966 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 59465us
I20260709 12:11:45.715750 15399 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.797479 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:45.785089 10582 rpcz_store.cc:276] 0709 12:11:44.727266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.727764 (+   498us) service_pool.cc:224] Handling call
0709 12:11:44.728186 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:44.728603 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:44.728636 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:44.768622 (+ 39986us) log.cc:844] Serialized 26 byte log entry
0709 12:11:45.388833 (+620211us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:45.388885 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.388962 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:45.784433 (+395471us) raft_consensus.cc:1685] finished
0709 12:11:45.784462 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.784613 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":287,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":16826,"thread_start_us":570447,"threads_started":1,"wal-append.queue_time_us":882468}
I20260709 12:11:45.801245 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:45.801664 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.803016 10582 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:45.797713 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.688225 15334 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.745991 14901 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:45.748152 15303 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:45.799137 15256 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.788374 15407 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.808856 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:45.809659 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.743826 15446 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:11:45.794145 15458 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.819969 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:45.833470 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:45.834856 10479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.001s	sys 0.048s
I20260709 12:11:45.798120 15362 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:45.805605 15289 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.002s
I20260709 12:11:45.835383 10584 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:45.880208 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:45.881309 10578 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:45.835769 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.824499 10429 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
W20260709 12:11:45.836740 10479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.001s	sys 0.048s
I20260709 12:11:45.807511 15395 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:45.808732 15302 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:11:45.842379 15453 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:11:45.844704 15399 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:45.892932 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.009s	sys 0.009s
I20260709 12:11:45.845291 15394 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.864251 14959 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:45.864861 15459 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.816416 15429 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:45.882557 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 562415us
I20260709 12:11:45.847944 15433 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:45.865783 15409 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:45.887409 15407 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:45.889612 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:45.892310 15289 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:45.893352 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.009s	sys 0.009s
W20260709 12:11:45.894416 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58729.6us
W20260709 12:11:45.895151 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 85568us
W20260709 12:11:45.898303 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 230627us
I20260709 12:11:45.816004 14901 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:45.906967 10584 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
W20260709 12:11:45.912952 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.910943 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:45.909821 15457 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.917924 10580 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:45.910337 15315 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:45.962878 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:45.890046 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:45.915552 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:45.920411 15326 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.673s	user 0.012s	sys 0.001s
I20260709 12:11:45.922120 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2779) took 1379 ms. Trace:
I20260709 12:11:45.918437 15433 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:45.923182 15334 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.924214 15397 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:45.925647 15460 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:11:45.926129 15342 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.000s
I20260709 12:11:45.926801 15303 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.928942 15455 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:45.864238 15464 consensus_queue.cc:1059] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 12:11:45.910527 15441 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:11:45.933254 15366 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:45.998695 15228 consensus_meta.cc:287] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.081s	user 0.016s	sys 0.001s
W20260709 12:11:45.955245 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44187.5us
W20260709 12:11:45.961383 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:45.962073 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.961988 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:45.965976 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.968374 15463 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:45.974404 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58941.9us
I20260709 12:11:45.975370 10585 rpcz_store.cc:276] 0709 12:11:44.543163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.611213 (+ 68050us) service_pool.cc:224] Handling call
0709 12:11:44.611598 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:44.611995 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:44.612027 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:44.651253 (+ 39226us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:44.663044 (+ 11791us) log.cc:844] Serialized 26 byte log entry
0709 12:11:45.487802 (+824758us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:45.487868 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.487970 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:45.921771 (+433801us) raft_consensus.cc:1685] finished
0709 12:11:45.921796 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.922047 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":393,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11294,"spinlock_wait_cycles":12224384,"thread_start_us":778645,"threads_started":1,"wal-append.queue_time_us":950569}
I20260709 12:11:45.982689 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:45.953569 15395 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.990648 15364 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:45.978351 15390 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:11:46.020669 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.975641 15433 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.981577 15352 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:45.995386 14901 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:45.988034 15467 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:45.996973 15464 raft_consensus.cc:3055] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:11:46.030834 15464 raft_consensus.cc:740] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:46.031909 15464 consensus_queue.cc:260] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.033406 15464 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:11:45.992748 15342 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.942363 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:45.980221 15289 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:46.036293 15289 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:46.036845 15289 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.012665 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:46.038782 15366 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:46.039088 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:46.039373 15366 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:46.042091 15455 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:46.042310 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:46.042605 15455 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:45.979305 15457 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.044150 10506 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:46.045521 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:46.047075 10436 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:46.010162 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.020319 15315 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:45.980614 15326 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:46.025221 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.979137 15459 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:11:46.047921 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:46.056335 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 114099us
I20260709 12:11:46.058498 11219 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:11:46.055989 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45906.1us
I20260709 12:11:46.058769 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:46.022078 10580 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:45.996119 15445 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.061735 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:46.069015 10438 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
W20260709 12:11:46.069169 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.072360 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.073307 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.073587 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.073251 10580 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:46.064730 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:46.073941 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.057682 10586 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:11:46.075835 10581 raft_consensus.cc:2368] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:46.076802 11219 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:46.066217 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:46.080395 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:46.081231 10434 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.073712 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.084358 10431 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:46.028285 15390 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:46.091454 10509 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:46.094647 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 83358.5us
I20260709 12:11:46.067361 15302 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:46.020852 15463 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.095378 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:46.078552 15470 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:11:46.098518 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.043591 15455 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.104429 15457 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:46.106017 15326 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.108470 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:46.109473 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.110882 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.096276 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:46.167956 10433 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:11:46.117309 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58641.5us
I20260709 12:11:46.169557 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.061693 15407 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:46.121701 10584 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.079296 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:46.175179 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 95962.2us
I20260709 12:11:46.176244 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:46.023654 15479 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:11:46.022080 15395 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.200924 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:46.163682 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46171.3us
I20260709 12:11:46.201421 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:46.026262 15458 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 12:11:46.109261 15309 tablet_copy_client.cc:538] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:11:46.187764 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 77022.8us
I20260709 12:11:46.216683 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.144027 10436 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:46.107187 15399 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:46.195909 10479 consensus_peers.cc:597] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:11:46.205003 15346 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.003s
I20260709 12:11:46.206212 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:46.201236 15459 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:46.208341 15357 raft_consensus.cc:3055] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:46.208092 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:46.215204 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:46.196125 15394 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:46.236613 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:46.248936 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:46.255882 10435 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:46.250164 10433 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:46.244326 10584 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
W20260709 12:11:46.244191 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 74777.7us
W20260709 12:11:46.249984 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 71126us
I20260709 12:11:46.236418 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:46.260260 10323 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e4cd5d330f694dacb199bd61266c8f5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 52bae2d2caf94d97b1049ecd4f82a2df is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:46.182070 15451 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:46.190135 15471 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:46.265697 10578 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 10: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:46.248337 10437 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:46.270172 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 68358.4us
I20260709 12:11:46.270404 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:46.189699 15447 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:46.056838 15467 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.044054 15366 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.258827 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.297230 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:46.200066 15478 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.273314 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:46.285269 10509 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:46.248010 15357 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:46.247408 15490 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:46.317826 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:46.318249 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:46.249800 15407 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:46.326206 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.327724 10581 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:46.328500 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.253628 15346 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.265058 15352 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.286386 15489 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:46.314695 15451 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:46.336253 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.352154 10586 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:46.269068 15471 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:46.296494 15459 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:11:46.337594 15476 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:11:46.268249 15315 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
W20260709 12:11:46.380951 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83822.1us
I20260709 12:11:46.394565 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:46.382848 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 124123us
W20260709 12:11:46.464365 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 227869us
I20260709 12:11:46.465691 10403 leader_election.cc:400] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:46.466107 10403 leader_election.cc:403] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:11:46.465837 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.385407 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.466913 10581 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.387084 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.249079 15399 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.311913 15447 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:46.463663 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.451136 10513 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:46.339778 10511 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
W20260709 12:11:46.385053 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 67333.1us
I20260709 12:11:46.471282 10578 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:11:46.471720 10578 pending_rounds.cc:85] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 8
I20260709 12:11:46.471988 10578 pending_rounds.cc:107] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:11:46.472313 10578 raft_consensus.cc:2889] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:46.475167 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:46.481786 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:46.485749 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.486021 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:46.381525 15452 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:11:46.384764 15407 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:11:46.497728 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 269725us
I20260709 12:11:46.376849 15494 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:46.388739 15342 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:46.390746 15362 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:46.529443 15362 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:46.530150 15362 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.533465 15399 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.534175 15399 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:11:46.535319 15499 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:46.480291 15416 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:11:46.538743 15346 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:46.540210 15483 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:11:46.542104 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.391302 15471 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:46.543628 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.544319 10587 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:46.544509 11220 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:11:46.543658 15447 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:46.545679 15445 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.321498 15478 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.495720 15256 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:46.505746 10403 leader_election.cc:400] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
W20260709 12:11:46.552059 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46514.1us
I20260709 12:11:46.553511 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:46.554469 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60887.3us
W20260709 12:11:46.507210 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.507889 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.503185 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.511791 10581 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:46.557114 10584 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:46.560467 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:46.557209 10582 raft_consensus.cc:2376] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13. Re-sending same reply.
I20260709 12:11:46.563779 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.564530 10585 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:46.512079 10583 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:46.517418 15407 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:46.517908 15342 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:46.357800 15490 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.512930 15494 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:46.389109 15451 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.527976 15395 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.374075 15390 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.528546 15463 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:46.551497 15256 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:46.560027 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:46.489502 15303 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12.
I20260709 12:11:46.383080 15489 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.555431 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:46.603953 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:46.629623 11217 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:46.606853 15446 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:11:46.605403 15397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.016s
W20260709 12:11:46.631912 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 127413us
W20260709 12:11:46.630373 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75076.9us
W20260709 12:11:46.574250 15464 consensus_meta.cc:287] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.529s	user 0.014s	sys 0.009s
I20260709 12:11:46.550825 15478 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:46.574048 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.562783 15223 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:46.627097 10506 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:46.580578 15447 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 12:11:46.634845 15397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.001s	sys 0.017s
I20260709 12:11:46.615355 15342 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:46.633591 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:46.649284 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 154019us
I20260709 12:11:46.581254 15352 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:11:46.605230 15357 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.632747 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:46.607846 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:46.656695 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 82798.6us
W20260709 12:11:46.673142 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.618203 15494 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:46.635733 15037 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:46.671347 10580 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 4.
I20260709 12:11:46.673861 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:46.673882 10585 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:46.677604 10403 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14.
I20260709 12:11:46.678249 10403 leader_election.cc:403] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:11:46.679906 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:46.680572 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47077.6us
W20260709 12:11:46.665068 15334 consensus_meta.cc:287] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.015s
W20260709 12:11:46.680469 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47814.8us
I20260709 12:11:46.636922 15463 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:11:46.683533 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:46.664816 15446 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:46.680335 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:46.685896 15256 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.656060 15223 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.685390 15351 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:46.676817 15037 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:46.731326 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47878.3us
I20260709 12:11:46.689071 15395 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:46.732519 15395 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:46.733526 15395 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.734010 15288 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:46.734606 15288 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:46.735934 15288 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:46.679258 14959 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.763s	user 0.014s	sys 0.004s
I20260709 12:11:46.739254 15037 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:46.745818 15289 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.002s
I20260709 12:11:46.768914 15489 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.773473 15364 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.691294 15451 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:46.688658 15493 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:11:46.693145 15309 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:11:46.754317 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74074.8us
W20260709 12:11:46.740126 15433 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.005s
I20260709 12:11:46.784850 15433 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:46.687191 15455 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:46.678402 15497 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:46.683167 15303 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14.
I20260709 12:11:46.741837 14901 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:46.791699 14901 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:46.745630 15391 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:46.774061 15364 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.780354 10579 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:46.751842 15351 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:46.751516 15467 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:46.739612 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57082.7us
I20260709 12:11:46.804271 15351 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.769466 15471 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:46.801240 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 249858us
W20260709 12:11:46.807446 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 144552us
I20260709 12:11:46.787380 15455 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:46.817349 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:46.812666 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:46.792347 15479 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.004s	sys 0.014s
I20260709 12:11:46.686149 15407 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.831514 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:46.866968 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.809291 14959 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:46.879761 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48406.1us
I20260709 12:11:46.776026 15289 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:46.884315 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53195.4us
I20260709 12:11:46.883270  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:46.800822 15391 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:46.887996 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 278579us
W20260709 12:11:46.887828 10553 leader_election.cc:336] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.863s (SENT)
I20260709 12:11:46.892315 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:46.896368 10479 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:46.847895 15479 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.004s	sys 0.014s
W20260709 12:11:46.907549 15508 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:11:46.907869 15223 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:46.892273 14959 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.903964 15433 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:46.911706 15509 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:11:46.914052 15471 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:46.916059 15342 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.917037 14901 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:46.885797 15504 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:11:46.872795 15451 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:46.886538 15507 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:46.950469 10511 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
W20260709 12:11:46.945662 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49401.6us
I20260709 12:11:46.949230 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:46.956321 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:46.918823 14901 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:46.953874 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:46.962569 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.943087 15515 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:11:46.918700 15455 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:46.983712 10508 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:46.944367 15507 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:46.985319 15507 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:46.985991 15366 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:46.986466 15366 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:46.984691 15342 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:46.988687 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 100920us
W20260709 12:11:46.991437 10553 leader_election.cc:336] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.879s (SENT)
I20260709 12:11:46.991489 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:46.992210 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:46.994134 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:47.003443 15364 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.007565 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:46.901005 15346 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:11:46.943506 15334 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:47.020447 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:46.943552 15416 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:46.949465 14959 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:46.984694 15514 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:11:46.986903 15366 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:47.026291 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.029955 10506 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:46.952118 15223 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:11:47.064782 10585 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:46.989358 15451 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:11:47.073765 15390 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:47.079315 14810 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:47.079835 14810 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:47.027244 15352 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:47.081815 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 125585us
I20260709 12:11:47.082965 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:47.024379 15416 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.028669 15479 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:47.010449 15362 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:47.084983 15485 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:11:47.068578 15490 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:47.120127 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.004844 15391 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.105054 15416 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:47.161490 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.171973 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:47.188081 15523 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:11:47.188882 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2566) took 1114 ms. Trace:
I20260709 12:11:47.189357 10580 rpcz_store.cc:276] 0709 12:11:46.074166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.118744 (+ 44578us) service_pool.cc:224] Handling call
0709 12:11:47.188618 (+1069874us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54001ca554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:47.188775 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999529856}
I20260709 12:11:47.168416 15289 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.104877 15352 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.080797 14810 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.168684 15397 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:47.218325 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 226713us
W20260709 12:11:47.223409 10553 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.897s (SENT)
I20260709 12:11:47.177923 15407 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.183161 15326 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:47.226081 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.156703 15288 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:11:47.196187 15475 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:11:47.235018 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 277939us
I20260709 12:11:47.348676 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:47.230473 15326 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.273372 15391 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:47.232609 15256 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.278416 15352 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:47.283238 15500 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:11:47.285481 15362 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:47.236529 15407 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.286095 15507 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.287166 15364 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:47.290136 15037 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.002s
I20260709 12:11:47.296200 15416 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:47.299889 15494 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.003s
W20260709 12:11:47.329756 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.358844 15326 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:47.350637 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.351449 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.352494 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.355494 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.378535 10431 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:47.378831 10434 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:11:47.358479 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2707) took 1287 ms. Trace:
I20260709 12:11:47.365212 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.379905 10433 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:47.366714 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:47.381356 11220 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:47.379668 10438 rpcz_store.cc:276] 0709 12:11:46.070773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.071090 (+   317us) service_pool.cc:224] Handling call
0709 12:11:46.071512 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:46.447796 (+376284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:46.447863 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:46.469621 (+ 21758us) log.cc:844] Serialized 26 byte log entry
0709 12:11:46.673438 (+203817us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:46.673477 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.345753 (+672276us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002657d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:47.345813 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:47.345871 (+    58us) raft_consensus.cc:1685] finished
0709 12:11:47.345893 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:47.358390 (+ 12497us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83910,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":25578,"spinlock_wait_cycles":419439360,"thread_start_us":224122,"threads_started":2,"wal-append.queue_time_us":187931}
I20260709 12:11:47.383909 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.384598 10429 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:47.369560 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 146658us
W20260709 12:11:47.385644 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.927s (SENT)
I20260709 12:11:47.386070 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.372162 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.372268 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.346269 15471 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.367189 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.390120 11212 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
W20260709 12:11:47.390318 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.361670 15533 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:47.389292 10512 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:47.393635 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:47.391599 10553 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.030s (SENT)
I20260709 12:11:47.394465 10581 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:47.392678 10580 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:11:47.394567 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.362998 15500 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:47.392618 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:47.395766 10581 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:11:47.402822 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:47.404217 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:47.398665 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.399086 10510 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:47.370867 15475 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:47.366107 15531 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.399482 10434 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:47.377482 15497 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.375885 15499 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.409060 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.406363 10586 raft_consensus.cc:2393] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 9: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:47.410429 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.377198 15498 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.411849 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:47.414354 10578 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:47.411906 10581 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:47.416280 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.417455 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.416589 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:47.423210 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.424034 15459 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:47.424587 15547 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.425336 15547 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.431572 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:47.433079 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.434083 10511 raft_consensus.cc:2368] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17.
I20260709 12:11:47.375397 15037 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.436801 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.437727 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.438886 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.445210 10513 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:47.364748 15256 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:47.446772 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.452589 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.452575 10509 raft_consensus.cc:606] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:47.453965 10429 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:47.377317 15364 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.458218 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.387249 15494 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.463590 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.448061 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.411403 15478 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:47.405458 15530 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:47.499430 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48452.6us
I20260709 12:11:47.499821 10508 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:47.498844 10513 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:47.500669 10429 raft_consensus.cc:2376] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 7. Re-sending same reply.
I20260709 12:11:47.500447 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.501849 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:47.504709 15366 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.506975 15364 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:47.404978 15471 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.511425 15416 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:47.511883 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 448811us
W20260709 12:11:47.513489 10579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2790) took 1715 ms (client timeout 2193 ms). Trace:
I20260709 12:11:47.519963 10504 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:47.523054 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:47.527624 10579 rpcz_store.cc:269] 0709 12:11:45.798191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.798328 (+   137us) service_pool.cc:224] Handling call
0709 12:11:47.513288 (+1714960us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b5400337054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:47.513410 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174280576}
I20260709 12:11:47.523335 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.528983 10430 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:47.507751 14901 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:47.528313 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:47.534924 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:47.538244 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.539743 10505 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:47.540946 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.428697 15499 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.557099 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:47.563558 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:47.569442 10579 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:47.544250 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.572503 10429 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:47.572589 10437 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:47.575354 10581 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:47.571385 15532 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:11:47.579092 15536 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:11:47.420475 15531 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.579663 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:47.586705 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:47.586962 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.587355 10581 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.428561 15498 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.592233 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.428733 15497 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.594925 10513 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
W20260709 12:11:47.599011 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.599838 10504 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:11:47.603188 10505 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:47.651648 10579 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:47.579125 15532 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.654497 10505 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:47.553138 15478 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.664371 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.668139 11214 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:47.595428 15416 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.594054 15352 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:47.543814 14901 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.690748 15463 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:11:47.664222 15532 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:47.692734 10581 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:47.669916 15528 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:47.705817 10431 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:47.716735 14901 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:47.716327 15558 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.717933 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.717803 15558 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:47.720964 15558 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:47.722538 15558 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:47.695837 15396 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:11:47.684614 15407 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.724223 15407 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:47.724798 15407 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.727002 15531 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:47.729727 15498 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.731443 15499 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.686898 15416 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.544142 15475 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.692569 15507 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.697428 15228 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:11:47.685840 15547 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:47.744225 10436 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:11:47.746069 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:47.759734 10511 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.747907 10505 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:47.748241 10504 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:11:47.752718 10434 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:47.669584 15478 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:47.753167 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:47.680639 15502 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:11:47.756795 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:47.740561 15542 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:11:47.775184 15507 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:47.754555 15471 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:47.778451 15522 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.574s	user 0.003s	sys 0.000s
I20260709 12:11:47.752311 15256 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:47.740008 15546 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:11:47.755968 15037 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.755115 14810 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.780519 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.782651 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.870388 11220 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:47.768357 15547 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:47.783969 10504 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:47.734917 15416 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:47.829214 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.830662 10436 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:47.748637 15494 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:47.769861 15407 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:47.890602 10585 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:11:47.881938 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:47.710740 15528 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.891851 11220 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:11:47.874552 15493 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:11:47.892657 10507 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:11:47.897416 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:47.898181 10507 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:47.900228 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:47.901005 10583 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:47.904078 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.904562 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:47.905566 10581 raft_consensus.cc:2368] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:47.907677 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.911919 11219 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:47.914913 11219 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:47.915583 10505 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:47.916132 10579 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:47.918710 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.878187 15366 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:47.876935 15549 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.922430 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.923184 11214 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:47.919606 10505 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:47.925537 10552 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:47.940204 10507 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:47.905215 15416 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.943286 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:47.947996 15498 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:47.952435 15326 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.003s
I20260709 12:11:47.917109 15407 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:47.960620 11214 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:47.882333 15547 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.976876 10511 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:47.952597 15533 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:47.981645 15533 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:47.980402 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 295579us
I20260709 12:11:47.983881 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:47.996181 10432 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:47.995124 15463 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.003579 10438 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:47.939492 15549 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.996219 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.035607 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:48.042625 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46501.5us
I20260709 12:11:48.046588 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2435) took 1098 ms. Trace:
I20260709 12:11:48.047122 10510 rpcz_store.cc:276] 0709 12:11:46.948388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.948512 (+   124us) service_pool.cc:224] Handling call
0709 12:11:46.948821 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:47.407699 (+458878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.046257 (+638558us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b540036e554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:48.046502 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034937600}
I20260709 12:11:47.994628 15478 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.003360 15256 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:48.080655 11219 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:48.021425 15395 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:11:47.982379 15326 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.982074 15533 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:48.087414 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.089184 11178 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:47.986052 15498 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.108494 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.089594 10438 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:48.095890 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.097126 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.097544 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:47.985157 15407 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:47.994498 15548 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 12:11:48.036966 15463 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.101353 15549 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.102478 15528 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:48.102764 15499 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:48.038964 15494 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:11:48.106861 15478 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.142843 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.148214 10585 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:47.971730 15416 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.027761 15497 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.115451 10504 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:48.116632 15498 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.110463 15569 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:11:48.117313 11214 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:48.108891 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.112025 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.119431 10510 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:48.118862 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:48.120103 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 138234us
W20260709 12:11:48.158197 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69087.4us
I20260709 12:11:48.158452 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.121642 10506 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:48.121608 10505 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:48.082149 15459 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:48.182260 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 70329.2us
I20260709 12:11:48.183202 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:48.125078 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.184551 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.100374 15552 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:48.126567 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.128991 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.117686 15502 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:48.113292 15256 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.107999 15570 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:48.137022 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.207234 10582 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:48.137945 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.140059 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.141508 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.209522 10583 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:48.144181 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.210500 10578 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:48.119246 15395 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:48.152001 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.212669 10585 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:48.157817 10507 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:48.159565 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.160825 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.164847 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.166034 15569 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:11:48.166855 15497 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:48.167630 15391 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.159763 15528 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.159952 15498 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.155303 15478 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.175879 15326 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.176829 15547 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:48.161922 15565 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:11:48.178840 15500 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.188493 10511 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:48.217832 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57101.3us
I20260709 12:11:48.190639 15575 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:48.224086 15391 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.188957 15459 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.213426 15256 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.208999 15570 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:48.228794 15548 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:48.251340 15548 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.251771 15548 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:48.212510 15574 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:48.222157 15502 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:48.181066 15532 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.253494 10403 leader_election.cc:400] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17.
W20260709 12:11:48.256405 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 98074.4us
I20260709 12:11:48.257393 10403 leader_election.cc:403] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:48.252632 15500 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:48.246141 15572 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:11:48.258850 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98122us
I20260709 12:11:48.259256 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.221472 15395 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.277320 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.250839 15478 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.278775 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.248117 15407 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.245945 15463 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:48.238585 15528 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.281095 10582 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:48.272392 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:48.274349 10432 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Committing config change with OpId 11.7: config changed from index -1 to 7, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:11:48.237768 15498 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.244578 15416 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.282763 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:48.332557 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49887.5us
I20260709 12:11:48.273805 15552 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17.
I20260709 12:11:48.256089 15570 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:48.274269 15502 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:48.310273 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 125809us
I20260709 12:11:48.340301 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.339854 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.337704 15582 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:11:48.300560 15497 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:48.281734 15547 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:48.335556 15554 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.329342 15464 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:48.309376 15549 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:48.353080 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2438) took 1398 ms. Trace:
I20260709 12:11:48.364260 15498 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.367074 15256 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.369047 15395 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.362794 15391 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:48.370016 15464 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:48.389840 15520 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.195s	user 0.004s	sys 0.000s
I20260709 12:11:48.389940 15464 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.390719 15519 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.391513 15519 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.370257 15549 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:48.392725 15549 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:48.393844 15519 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.345871 15459 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.345118 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.375804 10580 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:48.276854 15569 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:11:48.372241 15357 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:48.401963 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.383769 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.276582 15576 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 12:11:48.387058 10512 rpcz_store.cc:276] 0709 12:11:46.954598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.954901 (+   303us) service_pool.cc:224] Handling call
0709 12:11:46.955265 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:47.393955 (+438690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.352705 (+958750us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54001991d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:48.352990 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997657472}
I20260709 12:11:48.400023 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2737) took 1032 ms. Trace:
I20260709 12:11:48.422302 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.369199 15548 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.422825 10432 rpcz_store.cc:276] 0709 12:11:47.367831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.997494 (+629663us) service_pool.cc:224] Handling call
0709 12:11:47.998251 (+   757us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:48.037767 (+ 39516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:48.037804 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:48.112774 (+ 74970us) log.cc:844] Serialized 518 byte log entry
0709 12:11:48.274132 (+161358us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:48.396195 (+122063us) log.cc:844] Serialized 14 byte log entry
0709 12:11:48.399540 (+  3345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.399662 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:48.399726 (+    64us) raft_consensus.cc:1685] finished
0709 12:11:48.399745 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:48.399933 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63742,"raft.queue_time_us":52172,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":42763,"thread_start_us":101304,"threads_started":1,"wal-append.queue_time_us":222062}
I20260709 12:11:48.279706 15574 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 12:11:48.399225 15552 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.373366 15461 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:48.373709 15390 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:48.440002 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 167720us
I20260709 12:11:48.441869 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:48.427297 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.440791 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:48.365109 15497 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.346223 15532 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:48.436618 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 153407us
I20260709 12:11:48.410694 15407 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:48.423316 15464 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:48.360059 15302 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:48.444442 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.420804 15528 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.430097 15364 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:48.437625 15416 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.452581 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.454593 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2742) took 1080 ms. Trace:
I20260709 12:11:48.455335 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.464664 10583 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:48.440114 15461 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:48.447585 15497 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:48.428069 15584 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:11:48.431090 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 86076.7us
I20260709 12:11:48.430393 15391 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:11:48.459718 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.456203 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.464249 10431 rpcz_store.cc:276] 0709 12:11:47.373978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.118346 (+744368us) service_pool.cc:224] Handling call
0709 12:11:48.119479 (+  1133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:48.315499 (+196020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:48.315551 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:48.315617 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:48.315659 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.454234 (+138575us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400087ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:48.454305 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:48.454507 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450104064}
I20260709 12:11:48.464253 15429 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:48.436282 15581 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:11:48.447891 15394 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:11:48.444923 15326 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:11:48.448407 15390 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.452445 15533 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.471575 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.468839 11216 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:48.375732 15502 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.442612 15552 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.527212 10507 raft_consensus.cc:2410] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:11:48.469772 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.527590 10581 raft_consensus.cc:2410] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:11:48.479914 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:48.537822 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:48.528429 10513 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:48.538409 10511 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:48.471342 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:48.512928 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.540190 10511 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:48.537003 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.541265 10512 raft_consensus.cc:2393] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 10: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:48.541785 10511 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:48.450707 15582 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:48.477648 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:48.543855 11214 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:48.546116 10582 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:48.546790 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.453042 15366 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:48.547879 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:48.469509 11177 leader_election.cc:400] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:48.551339 11177 leader_election.cc:403] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:11:48.530920 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.553634 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:48.556907 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:48.557699 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 88355.5us
I20260709 12:11:48.559425 10580 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:11:48.518909 15537 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.951s	user 0.001s	sys 0.001s
I20260709 12:11:48.564800 10511 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:48.567459 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.559098 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:48.464438 15302 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:48.467504 15362 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:48.526924 15429 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:48.506453 15334 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:48.579722 10578 raft_consensus.cc:2368] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:48.482684 15521 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:48.471825 15364 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:48.473565 15461 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:48.602571 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.604610 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:48.579334 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.545744 15532 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.546041 15478 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:48.610513 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 166175us
I20260709 12:11:48.555751 15557 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.467665 15528 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 12:11:48.616051 15548 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.565698 15586 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:48.467020 15576 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.115s
I20260709 12:11:48.560344 15394 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:48.670315 10586 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:48.469691 15416 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.566306 15582 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:48.629700 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:48.664645 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:48.638262 10511 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:48.638649 10513 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:48.665392 15478 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.656029 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.660568 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:48.681741 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:48.556264 15499 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:11:48.602267 15587 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:48.656141 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:48.639293 10512 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:48.562698 15366 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.679247 10582 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:48.627581 15547 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.627702 15334 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:48.670464 15521 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:48.707504 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:48.708070 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 154566us
I20260709 12:11:48.692051 10509 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:11:48.695406 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:48.709048 10511 raft_consensus.cc:606] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:11:48.709425 10511 raft_consensus.cc:614] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:48.703562 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.703363 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.704334 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.705269 10438 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:48.713092 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.713265 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 731) took 1617 ms. Trace:
I20260709 12:11:48.711745 10432 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:48.622937 15570 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.706028 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.689229 15326 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.714937 10430 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:48.705850 10513 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:48.627321 15362 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.685161 15596 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.667986 15532 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.749128 10432 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:48.713627 11217 rpcz_store.cc:276] 0709 12:11:47.082023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.082750 (+   727us) service_pool.cc:224] Handling call
0709 12:11:47.083098 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:47.083538 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:47.083572 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:47.084220 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:11:47.369961 (+285741us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:47.370001 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.370066 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:48.518081 (+1148015us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:48.683252 (+165171us) raft_consensus.cc:1685] finished
0709 12:11:48.683279 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:48.699223 (+ 15944us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":20725,"spinlock_wait_cycles":8164736,"thread_start_us":215457,"threads_started":1,"wal-append.queue_time_us":349593}
I20260709 12:11:48.749671 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.750825 10583 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:48.716755 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.744608 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.716995 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.688987 15587 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:48.716581 14810 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:11:48.767719 15587 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:48.709213 15395 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:48.769423 15395 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.756886 15334 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.757555 10430 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:48.760149 15362 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.761477 15552 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.762154 15570 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.762579 11220 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 7.
I20260709 12:11:48.758538 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:48.793640 10513 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.794080 10513 raft_consensus.cc:614] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:48.694623 15547 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:48.756865 15498 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.740075 15594 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:48.790409 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:48.805133 10583 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:48.716343 15591 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:48.795651 15547 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.694139 15584 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
W20260709 12:11:48.816882 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 107564us
I20260709 12:11:48.742357 15567 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:11:48.822597 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.823359 10429 raft_consensus.cc:2376] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14. Re-sending same reply.
I20260709 12:11:48.810484 15570 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.741540 15429 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.825573 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.830731 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.831825 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.832834 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.834518 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.836570 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:48.836761 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.838204 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.844233 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.741254 15364 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.844866 10513 raft_consensus.cc:606] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.744161 15302 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.818881 15582 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.810642 15498 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.818625 15591 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:48.867662 11177 leader_election.cc:400] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:48.865310 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.853082 10429 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:48.877733 11177 leader_election.cc:403] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:48.860234 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.873579 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:48.881007 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.881889 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:48.882202 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.883304 10434 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:48.884406 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.877982 10508 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:11:48.886111 10434 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:48.886207 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.879381 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.892586 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.894819 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.896245 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.897023 10505 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:48.891891 15581 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:48.898667 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2725) took 1549 ms. Trace:
I20260709 12:11:48.899142 10437 rpcz_store.cc:276] 0709 12:11:47.349676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.351188 (+  1512us) service_pool.cc:224] Handling call
0709 12:11:48.898450 (+1547262us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540034a654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:48.898594 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445374720}
I20260709 12:11:48.900985 10437 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:48.902251 10437 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:48.903924 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:48.876169 15593 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:11:48.787376 15394 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:48.860795 15597 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:11:48.907263 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2579) took 1475 ms. Trace:
I20260709 12:11:48.870028 15459 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:48.908972 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:48.829973 15533 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.754758 15596 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.910861 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.856545 15416 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.846418 15429 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:48.913599 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.914804 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:48.847961 15395 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.915913 10506 rpcz_store.cc:276] 0709 12:11:47.431804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.433640 (+  1836us) service_pool.cc:224] Handling call
0709 12:11:47.501487 (+ 67847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:48.162276 (+660789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.906966 (+744690us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54002c6ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:48.907189 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425680128}
I20260709 12:11:48.918529 10505 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:11:48.918670 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:11:48.928293 10508 raft_consensus.cc:606] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:48.928687 10508 raft_consensus.cc:614] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:48.920912 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:48.921766 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:48.883090 15390 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:48.858028 15532 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.940568 10435 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.941901 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.944372 10434 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:48.945505 10438 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:48.946749 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.923732 15459 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:48.951032 10583 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:48.815472 15594 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.952924 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.929140 15478 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:48.948104 15494 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:48.951601 15499 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:48.969964 15499 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.957865 15591 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:48.970904 15309 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=1 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:11:48.971869 15550 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.217s	user 0.005s	sys 0.000s
I20260709 12:11:48.972193 15309 tablet_bootstrap.cc:492] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:11:48.972522 15499 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.954303 15552 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:48.833809 15521 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.973387 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2573) took 1580 ms. Trace:
I20260709 12:11:48.960564 15596 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:48.944648 15395 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:48.954619 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:48.957151 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:48.965154 15569 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:48.975596 10433 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:48.975150 10587 rpcz_store.cc:276] 0709 12:11:47.392792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.395312 (+  2520us) service_pool.cc:224] Handling call
0709 12:11:47.395613 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:47.401330 (+  5717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:47.401364 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:47.416261 (+ 14897us) log.cc:844] Serialized 26 byte log entry
0709 12:11:47.657628 (+241367us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:47.657681 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.657753 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:48.971176 (+1313423us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:48.973041 (+  1865us) raft_consensus.cc:1685] finished
0709 12:11:48.973065 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:48.973316 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5290,"raft.queue_time_us":5827,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7588,"spinlock_wait_cycles":5303552,"thread_start_us":146171,"threads_started":1,"wal-append.queue_time_us":251758}
I20260709 12:11:48.978240 15395 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.978039 15569 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:48.979823 15569 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:48.978837 15395 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:48.983162 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.984794 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.986554 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.990029 10578 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 12:11:48.994059 15594 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:48.994756 15601 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:49.013091 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:49.023777 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:48.996405 15366 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:48.998791 15526 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.731s	user 0.004s	sys 0.000s
I20260709 12:11:49.027565 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:48.990644 15499 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:49.001149 15558 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:48.986860 10435 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:49.006301 10433 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:49.007663 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:49.008392 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:49.023888 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:49.037824 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:48.997336  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:49.039624 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:49.032476 10513 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:11:49.033324 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:49.023257 15309 ts_tablet_manager.cc:1403] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 2.330s	user 0.049s	sys 0.022s
I20260709 12:11:48.988446 15596 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:48.991304 15552 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:49.012871 15458 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.051136 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:49.024930 15475 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.980826 15494 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.029202 15498 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.042394 15558 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.035717 15499 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:49.031407 15357 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:11:49.053871 15519 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:49.053609 15407 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:49.074154 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2814) took 2084 ms. Trace:
I20260709 12:11:49.075250 11216 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
W20260709 12:11:49.076879 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 381105us
I20260709 12:11:49.036204 15576 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:11:49.053360 15552 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.062670 15416 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:49.083850 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56417us
I20260709 12:11:49.081611 10584 rpcz_store.cc:276] 0709 12:11:46.989967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.990103 (+   136us) service_pool.cc:224] Handling call
0709 12:11:47.004476 (+ 14373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:47.004729 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:47.004763 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:47.005253 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:11:47.107076 (+101823us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:47.107138 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.107240 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:48.998150 (+1890910us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:49.064705 (+ 66555us) raft_consensus.cc:1685] finished
0709 12:11:49.064731 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:49.074057 (+  9326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16923904,"thread_start_us":101454,"threads_started":1,"wal-append.queue_time_us":232323}
W20260709 12:11:49.089257 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49777.8us
I20260709 12:11:49.090655 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:49.067019 15558 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:49.067390 15601 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:49.078910 15499 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:49.076908 15498 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:11:49.086163 15598 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:11:49.141629 15606 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
I20260709 12:11:49.104681 15228 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:49.104844 15586 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:49.148155 15228 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:49.148329 15586 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.148897 15586 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:49.149223 15228 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.081353 15407 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:49.150612 15228 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:49.105265 15530 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.105835 15614 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:49.105971 14901 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.080751 15519 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.113926 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:49.151836 15530 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.160292 15490 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:49.160745 15490 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.095713 15601 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.161180 15490 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:49.161237 15601 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:49.161628 15530 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.151870 15614 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:49.163158 15326 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:49.163405 15614 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.165215 15614 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:49.116916 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:49.167200 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.117419 15572 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.087450 15528 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:11:49.178567 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:49.166314 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 72938.3us
I20260709 12:11:49.120299 15309 raft_consensus.cc:359] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:49.136096 15494 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.088773 15552 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:49.046725 15459 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.063994 15458 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.198102 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.066327 15475 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.155448 15467 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:11:49.201179 15459 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:49.201913 15475 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.139915 15461 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.203735 15581 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:49.204133 15461 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.141459 15587 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.206427 15587 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.207242 10582 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:49.183914 15455 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 12:11:49.145658 15497 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
W20260709 12:11:49.208122 15557 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.002s
I20260709 12:11:49.208225 15497 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.208685 15557 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:49.178913 14810 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.102825 15416 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:49.169792 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.210969 14810 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.211665 10580 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:49.211764 15416 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.212846 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.213476 14810 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.171761 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.177603 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:49.155864 15549 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.763s	user 0.015s	sys 0.000s
W20260709 12:11:49.152541 15464 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.695s	user 0.014s	sys 0.000s
I20260709 12:11:49.189842 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.100950 15364 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:49.152664 15519 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:49.191502 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 90082.4us
I20260709 12:11:49.232589 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:49.231273 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.234637 10583 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:11:49.193293 15395 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.199179 11214 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:49.150942 15407 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:49.219962 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44778.9us
I20260709 12:11:49.224750 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.214193 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.211185 10436 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:49.239109 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:49.235987 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.210553 15497 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.187686 15572 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.234567 15364 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.248899 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.250487 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.258152 10579 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:49.267582 10587 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:49.191264 15309 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:11:49.152328 14901 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.264196 10580 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:49.236671 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:49.281718 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.288786 15490 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:49.292158 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 268380us
I20260709 12:11:49.293573 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:49.232081 15362 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:11:49.239076 15610 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:49.210493 15547 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:49.248947 15352 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:49.304697 10580 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:49.264173 15256 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:11:49.318181 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 81649.7us
I20260709 12:11:49.235558 15549 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.294528 15364 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:49.298125 15407 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:49.319934 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:49.296403 15497 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.263867 15458 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.346648 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:49.315425 15570 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:49.363095 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:49.317406 15610 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.372648 15610 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.334949 15590 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:49.375507 10581 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:49.374922 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:49.301846 15475 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:11:49.382053 10586 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:49.397514 10578 raft_consensus.cc:2410] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:49.322274 15352 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:49.399861 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 100170us
I20260709 12:11:49.400337 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:49.403637 10479 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:49.336730 15256 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.376868 15557 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:49.401093 15605 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:11:49.409792 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.396468 15590 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:49.436106 10432 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:49.383421 15416 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:49.348572 15497 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.420547 15352 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:49.421602 15612 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:11:49.413986 15610 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:49.398741 15563 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:49.415654 15557 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:49.465744 15552 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.467525 15395 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.470000 15552 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.475378 15621 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.401675 15570 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:49.333583 15309 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:49.480846 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:49.481871 11214 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:49.467257 15524 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:11:49.410861 15362 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:49.409608 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:49.407140 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66211.6us
I20260709 12:11:49.485697 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:49.476076 15621 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.479080 15557 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:49.504441 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 94913us
I20260709 12:11:49.478425 15416 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:49.484054 15490 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.551467 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:49.553408 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:49.511934 15394 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:49.511607 15362 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:11:49.568832 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:49.573261 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:49.576857 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:49.580935 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:49.582176 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.583813 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:49.499502 15558 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.586686 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:49.586001 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:49.587687 11219 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:49.577584 15352 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.591599 15352 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.594239 15352 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.595378 15498 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:49.595541 15309 ts_tablet_manager.cc:1434] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.543s	user 0.010s	sys 0.003s
I20260709 12:11:49.595853 15498 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:49.595973 15309 ts_tablet_manager.cc:1467] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Tablet startup took 548ms
I20260709 12:11:49.596956 15498 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:49.571754 15394 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.598434 15498 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:49.598732 15394 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:49.601565 15616 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:11:49.601969 15624 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:11:49.604074 15552 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.604715 15552 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.609074 11212 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:11:49.609762 11212 pending_rounds.cc:85] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 10
I20260709 12:11:49.610160 11212 pending_rounds.cc:107] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:11:49.610559 11212 raft_consensus.cc:2889] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:49.564445 15547 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:49.621871 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:49.622932 11214 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:11:49.623596 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:49.622254 15584 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:49.627687 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:49.624614 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:49.618034 15596 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:11:49.589393 15572 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:49.615934 15326 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:49.615670 15524 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:49.647545 15352 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:11:49.671700 15620 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:11:49.672173 15518 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:11:49.673162 15619 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:11:49.673605 15625 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:11:49.681164 15362 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:49.684986 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:49.561734 15590 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:50.083423 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 532003us
I20260709 12:11:49.929461 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2747) took 1226 ms. Trace:
W20260709 12:11:49.596386 15309 ts_tablet_manager.cc:1469] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Trace:
0709 12:11:41.120659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.121004 (+   345us) service_pool.cc:224] Handling call
0709 12:11:41.564491 (+443487us) ts_tablet_manager.cc:934] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:11:42.283361 (+718870us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:11:42.838251 (+554890us) tablet_metadata.cc:665] Metadata flushed
0709 12:11:42.838435 (+   184us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:11:43.393738 (+555303us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:11:43.461621 (+ 67883us) tablet_replica.cc:206] Creating consensus instance
0709 12:11:43.498692 (+ 37071us) ts_tablet_manager.cc:989] Replica registered.
0709 12:11:43.498772 (+    80us) inbound_call.cc:177] Queueing success response
0709 12:11:46.692672 (+3193900us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:11:49.052200 (+2359528us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:11:49.104614 (+ 52414us) tablet_replica.cc:261] Starting instrumentation
0709 12:11:49.105388 (+   774us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":98890,"raft.queue_time_us":4685,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":10038,"tablet-copy.queue_time_us":329058,"tablet-download-pool.queue_time_us":180,"tablet-download-pool.run_cpu_time_us":1400,"tablet-download-pool.run_wall_time_us":2465590,"thread_start_us":187629,"threads_started":2}
I20260709 12:11:50.062942 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.084751 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:50.095930 10512 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
W20260709 12:11:50.085927 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 458303us
W20260709 12:11:49.754763 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.086606 10508 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:50.098600 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.099565 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:50.101262 10478 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:49.687558 15467 raft_consensus.cc:993] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:11:49.780114 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:49.685834 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.084335 15518 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:50.128031 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:49.835090 15499 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.005s
W20260709 12:11:49.918478 15364 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.576s	user 0.004s	sys 0.006s
W20260709 12:11:50.092689 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 539370us
W20260709 12:11:49.691687 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67174.8us
I20260709 12:11:50.093215 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:50.137670 10513 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:50.137974 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:49.913383 15629 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:49.891963 15497 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.850615 15549 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.084861 10436 rpcz_store.cc:276] 0709 12:11:48.703500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.703945 (+   445us) service_pool.cc:224] Handling call
0709 12:11:49.929197 (+1225252us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54000875d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:49.929369 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147290240}
I20260709 12:11:49.639508 15396 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:11:50.125618 10478 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:49.924798 15601 consensus_meta.cc:287] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.001s
W20260709 12:11:49.835404 15586 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.686s	user 0.008s	sys 0.008s
W20260709 12:11:49.836234 15614 consensus_meta.cc:287] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.671s	user 0.017s	sys 0.000s
I20260709 12:11:50.126396 15584 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:50.065852 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 871) took 1322 ms. Trace:
I20260709 12:11:49.930475 15366 raft_consensus.cc:993] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
W20260709 12:11:50.132431 11220 consensus_meta.cc:287] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.002s
I20260709 12:11:50.086005 15569 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:50.090935 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.748610 15548 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.140801 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:49.896236 15607 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.816s	user 0.000s	sys 0.008s
I20260709 12:11:49.805338 15256 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:49.928110 15558 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.093869 10505 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:50.098752 15334 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:49.735253 15627 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:49.920511 15621 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.150235 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:49.692090 15416 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.084698 15326 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:50.126452 15624 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:11:49.859441 15618 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 12:11:49.905972 15547 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.151847 15629 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.199651 15629 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.201431 15395 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:50.201918 15395 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:50.127267 15591 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:50.127080 15572 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.161994 11220 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:50.168658 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.148267 15524 raft_consensus.cc:1081] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:11:50.161258 15630 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:50.151759 15628 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.154805 15574 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.192046 15326 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:50.173663 15548 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:49.688268 15394 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.160279 15584 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:50.181031 15627 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:50.161410 15352 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.146899 15499 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.192584 15624 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:50.186560 15625 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:11:50.179405 15475 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:50.160398 15601 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.164229 15586 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.108037 15362 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:50.225451 15519 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.974s	user 0.015s	sys 0.001s
W20260709 12:11:50.239701 15557 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.005s
I20260709 12:11:50.231807 15631 raft_consensus.cc:1081] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:11:50.202811 15395 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.231156 15628 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.249517 15394 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.235106 15584 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:50.189790 15518 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.215178 15547 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.239558 15624 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:11:50.231314 15574 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.227628 15572 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:50.254297 15519 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:50.334157 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:50.419448 15521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.432s	user 0.007s	sys 0.004s
W20260709 12:11:50.420073 15521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.440s	user 0.007s	sys 0.004s
W20260709 12:11:50.173354 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47824.9us
W20260709 12:11:50.340576 10435 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2744) took 1637 ms (client timeout 1614 ms). Trace:
I20260709 12:11:50.364079  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:50.365234 10331 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:50.182126 10508 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:50.163965 10513 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:50.147766 15364 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.369451 10509 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:50.169593 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:50.370018 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:50.372537 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.255795 15557 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.101262 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:50.169889 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.166646 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.160521 10582 rpcz_store.cc:276] 0709 12:11:48.743579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.757082 (+ 13503us) service_pool.cc:224] Handling call
0709 12:11:48.757434 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.207723 (+450289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.065430 (+857707us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54003f02d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:50.065748 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033910144}
I20260709 12:11:50.374374 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2593) took 1204 ms. Trace:
I20260709 12:11:50.236090 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.230415 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.167526 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:50.377501 11217 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:50.288398 15601 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.245191 15627 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.290087 15628 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:50.424053 10435 rpcz_store.cc:269] 0709 12:11:48.702450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.702582 (+   132us) service_pool.cc:224] Handling call
0709 12:11:50.339807 (+1637225us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54003539d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:50.340152 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424881024}
W20260709 12:11:50.413797 15582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.419s	user 0.011s	sys 0.005s
I20260709 12:11:50.416250 15549 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.240639 15352 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.428617 10432 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: [10.6-11.7]
I20260709 12:11:50.432700 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.436977 10431 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:50.449501 15396 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:50.444798 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.449016 15490 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.435549 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:50.494010 15557 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.422963 15572 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:50.429539 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.456634 10587 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:11:50.453222 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.425438 15499 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.455622 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.450640 10581 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:50.423717 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.434621 15364 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.445266 15497 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.430572 15362 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.508212 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.512210 15334 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:11:50.513049 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.419515 15554 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.513533 14901 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:50.514053 15471 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.443769 15548 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.454174 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:50.270900 15395 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:50.453444 10584 rpcz_store.cc:276] 0709 12:11:49.170408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.170582 (+   174us) service_pool.cc:224] Handling call
0709 12:11:49.189646 (+ 19064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:49.195213 (+  5567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:49.195248 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:49.195831 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:11:49.399506 (+203675us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:49.399559 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.399648 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:50.373612 (+973964us) raft_consensus.cc:1685] finished
0709 12:11:50.373660 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:50.374060 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13260,"raft.queue_time_us":5337,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":169,"thread_start_us":203222,"threads_started":1,"wal-append.queue_time_us":243359}
I20260709 12:11:50.519788 15615 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:50.451598 15629 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:50.555713 10433 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:50.448521 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 347134us
I20260709 12:11:50.557775 15549 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:50.558945 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2846) took 1379 ms. Trace:
I20260709 12:11:50.559644 15490 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:50.535557 15582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.541s	user 0.012s	sys 0.005s
I20260709 12:11:50.559484 10583 rpcz_store.cc:276] 0709 12:11:49.179682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.302204 (+122522us) service_pool.cc:224] Handling call
0709 12:11:49.319922 (+ 17718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.320147 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:49.320421 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 12:11:49.605874 (+285453us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b18003833e0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:50.167987 (+562113us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003833e0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c926874bc 00007f2c8f9d548b 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f23a
0709 12:11:50.558136 (+390149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:50.558280 (+   144us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:50.558415 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.558529 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:50.558716 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6898,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":22552,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":4048,"spinlock_wait_cycles":67981952,"thread_start_us":406203,"threads_started":2,"wal-append.queue_time_us":25112}
W20260709 12:11:50.576160 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 122409us
I20260709 12:11:50.454802 15638 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:11:50.580888 15638 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:50.581561 15638 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:50.445031 15518 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.588006 15615 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.588882 15615 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.591447 15458 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:50.591681 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.592583 10505 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:50.592764 10435 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:50.571337 10438 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:50.593863 10430 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
W20260709 12:11:50.591670 15643 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.177s	user 0.000s	sys 0.002s
I20260709 12:11:50.547925 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.595930 15352 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:50.596158 10586 raft_consensus.cc:2368] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:50.572178 15396 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.591990 15458 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:50.598763 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.573519 10582 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:50.600245 15458 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.602210 15458 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:50.593196 15614 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.570438 15490 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:50.611863 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54114.5us
I20260709 12:11:50.616825 15574 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:50.617233 10579 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:50.620795 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57890 (request call id 752) took 1135 ms. Trace:
I20260709 12:11:50.621363 11217 rpcz_store.cc:276] 0709 12:11:49.485137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.485324 (+   187us) service_pool.cc:224] Handling call
0709 12:11:50.620467 (+1135143us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b5400821c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:50.620669 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033044224}
I20260709 12:11:50.598845 15396 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.628504 14901 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:50.631088 15554 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.629480 15471 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.634445 15554 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.636461 15471 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.640123 15638 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.658035 10436 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:50.654806 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:50.679816 10434 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:50.655227 15572 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:11:50.649729 15626 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.978s	user 0.000s	sys 0.003s
I20260709 12:11:50.647851 15629 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:50.686148 15498 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.087s	user 0.014s	sys 0.012s
I20260709 12:11:50.657588 15519 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:50.705933 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91493.8us
I20260709 12:11:50.559317 15625 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.178s
W20260709 12:11:50.684185 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 360316us
I20260709 12:11:50.713461 11214 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:50.603617 15614 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.690058 15572 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:50.733186 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.734746 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:50.736413 11177 leader_election.cc:400] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:50.736701 11177 leader_election.cc:403] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:11:50.737785 15459 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:50.738237 15459 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:50.738613 15459 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:50.720557 15394 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:50.740294 15394 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:50.741714 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:50.742177 15642 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:11:50.743060 15498 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
W20260709 12:11:50.747232 15641 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.160s	user 0.002s	sys 0.000s
I20260709 12:11:50.755429 15586 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.757941 15552 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.768388 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.762859 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.781412 14901 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.655460 15497 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.763559 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.794852 15547 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.764868 15572 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:50.425752  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:50.823369 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.823464 10584 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 12:11:50.831669 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.835078 10331 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 15, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:50.839378 10585 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:50.837957 10331 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 13, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:50.842123 10331 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:50.852421 15366 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.853480 15366 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.861282 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:50.862293 10436 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:50.862753 15549 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.864816 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2849) took 1682 ms. Trace:
I20260709 12:11:50.865556 15366 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.865355 10580 rpcz_store.cc:276] 0709 12:11:49.182237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.313272 (+131035us) service_pool.cc:224] Handling call
0709 12:11:49.313746 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:49.314247 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:49.314291 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:49.323048 (+  8757us) log.cc:844] Serialized 26 byte log entry
0709 12:11:49.512768 (+189720us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:49.512823 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.512937 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:50.625604 (+1112667us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:50.864153 (+238549us) raft_consensus.cc:1685] finished
0709 12:11:50.864178 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:50.864577 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":12597,"spinlock_wait_cycles":481118464,"thread_start_us":138366,"threads_started":1,"wal-append.queue_time_us":234132}
I20260709 12:11:50.822587 14901 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.868957 15589 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:50.869791 15589 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:50.872457 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 539827us
I20260709 12:11:50.872859 15629 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.873406 15497 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.875669 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.876009 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.876910 10508 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:50.877424 10581 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:50.887678 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.844134 10331 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 14, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 14 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:50.888984 15589 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.889351 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.870661 14901 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:50.899860 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.885854 15394 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:50.925501 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:50.925837 15569 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.003s
I20260709 12:11:50.926595 15582 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.906728 10513 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:50.904176 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.927273 15582 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.907056 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.929170 10508 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:50.929266 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.907706 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.907757 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:50.931341 10510 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:50.911545 15326 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:50.933951 15646 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 12:11:50.934824 15326 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:50.912499 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48583.1us
I20260709 12:11:50.935267 15518 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:50.936582 15518 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.937852 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:50.941007 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:50.941776 11219 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:50.944607 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.945556 10580 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:50.946364 15497 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:50.947291 15497 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:50.947563 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.948520 10584 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:50.950084 15497 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.951556 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:50.902837 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:50.952878 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57772.2us
I20260709 12:11:50.953888 15459 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:50.923400 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.967553 10578 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:50.884416 15615 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.968261 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.968879 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.969714 10578 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:50.969889 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.969384 10331 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 11, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 11 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:50.970522 10583 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:11:50.969198 10585 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:50.971901 10331 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:50.975039 15582 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:50.977097 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51813.5us
W20260709 12:11:50.915122 11178 leader_election.cc:336] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.842s (SENT)
W20260709 12:11:50.980199 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 266868us
W20260709 12:11:50.982163 15407 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.559s	user 0.019s	sys 0.007s
W20260709 12:11:50.982829 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.957463 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:50.916489 15614 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:50.927276 14901 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:11:50.960453 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.987480 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:50.955137 15458 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:50.992197 10331 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 13, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 13 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:50.995438 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:50.996289 10510 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:50.995286 10331 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 13, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:50.993651 15518 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:50.988978 14901 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.007472 14901 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.009390 10507 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:11:50.989132 15458 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.007755 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.011263 15458 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.011515 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.015861 10505 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:51.022658 10584 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:51.002692 15459 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.013490 10513 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:51.014269 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:51.030244 11219 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:51.030320 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.031157 10585 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.031467 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:51.033356 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.031870 10581 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:50.985423 15497 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.036242 15497 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.043154 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.048844 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:51.049642 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.050293 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.051172 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.051605 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.053198 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.055528 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:51.056226 15584 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.782s	user 0.017s	sys 0.000s
I20260709 12:11:51.056035 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.056963 10580 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:51.056806 15584 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.056433 10505 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:51.058442 15572 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.059773 10584 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:51.063758 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.060946 10404 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:51.064693 10508 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:51.065317 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:51.066557 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.067368 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.068356 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:51.072829 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.079495 10505 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:51.082420 15627 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:51.084335 15624 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.807s	user 0.013s	sys 0.002s
I20260709 12:11:51.085204 15627 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:51.084940 15624 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.085772 15627 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.086522 10435 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:11:51.087311 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.086522 10431 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:51.092036 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:51.092312 10432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:11:51.089046 15648 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:51.093135 10432 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:51.093611 10432 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.093786 15648 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.059763 15547 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:51.073526 15334 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.061040 15518 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.098819 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:51.108016 15499 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:51.108471 15499 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.108837 15499 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:51.103655 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 57816.8us
I20260709 12:11:51.121786 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.122748 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.123148 15645 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:51.103940 10434 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 10.
I20260709 12:11:51.124241 10580 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:11:51.124879 10580 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:51.124737 15518 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.125370 10580 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:51.126049 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:51.126968 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:51.138670 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.146809 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:51.116390 10479 connection.cc:256] RPC call timeout handler was delayed by -1.6266821539998091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 12:11:51.149938 15459 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:51.169706 11219 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Advancing to term 13
I20260709 12:11:51.169919 15364 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:51.131165 15547 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.170687 15364 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.172228 15497 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.131762 15552 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.169600 10586 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
W20260709 12:11:51.175087 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 141187us
I20260709 12:11:51.145511 15334 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.188122 15558 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.188895 15558 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.177147 10435 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:51.192397 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.176393 10510 raft_consensus.cc:2410] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:11:51.188351 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:51.191845 11177 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.773s (SENT) [suppressed 14 similar messages]
W20260709 12:11:51.196949 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46476.5us
I20260709 12:11:51.166246 15518 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:51.175477 15654 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:11:51.205226 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.212105 15624 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:51.213174 15547 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.213057 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.214264 15334 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.158385 15582 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:51.234495 10585 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:51.235978 10436 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:51.237498 10586 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:11:51.255654 11217 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:51.178162 15648 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.169729 15459 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:51.175446 15572 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:51.258600 10587 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:51.276115 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.280649 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.283723 10510 raft_consensus.cc:2368] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:11:51.202690 15645 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:51.258870 15657 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:11:51.295346 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.257898 15582 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.293643 15615 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.309907 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.307997 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:51.281322 15459 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:51.294360 15647 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:11:51.311873 15582 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:51.226379 15458 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:51.325565 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46333.7us
I20260709 12:11:51.323289 15467 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:51.288197 15584 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.337859 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.338502 15458 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.339432 11214 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:11:51.330365 10430 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:51.341949 10510 raft_consensus.cc:2368] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 10. Current term is 11.
I20260709 12:11:51.344455 15628 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:51.285189 15558 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:51.334132 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 145941us
I20260709 12:11:51.348012 10511 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:51.350185 15459 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.336709 15467 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.357095 15467 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.357113 15407 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:51.351745 15471 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:51.360244 15471 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.362056 15364 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.362879 15471 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.286903 15548 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:11:51.303220 15552 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.335543 10513 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:51.364630 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.365412 15552 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:51.365739 15471 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:51.368094 15459 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.368710 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:51.375241 15662 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:11:51.393934 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:51.411150 10435 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:51.390538 15645 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:51.415246 15648 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:51.385890 15615 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.416074 15648 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:51.417271 15648 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.418355 15615 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.419643 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.422101 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:51.423699 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:51.424255 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:51.425058 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:51.425150 15471 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:51.383384 15552 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:11:51.426290 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:51.520730 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.521832 15552 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:11:51.426582 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.427162 15645 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:51.525038 15645 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:51.431633 15655 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 12:11:51.434675 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.527666 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.528664 10510 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:11:51.439128 11216 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:51.448137 15660 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:11:51.535248 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.537058 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.538292 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.448688 15549 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:51.540148 15549 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:51.540587 15549 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:51.541718 15549 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.457757 10513 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:51.459281 10432 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:51.544837 10432 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.545055 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.545874 10431 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:51.461946 15582 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.547080 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.547994 11215 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:51.549082 15582 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.549584 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:51.550084 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.550886 10505 raft_consensus.cc:2368] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:11:51.551451 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.472208 10585 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:51.553310 15552 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:51.554381 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.554152 15552 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.538908 10578 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:51.555056 10585 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:11:51.557165 15552 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:51.460870 10336 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 234) took 22395 ms (client timeout 15000 ms). Trace:
I20260709 12:11:51.560887 15549 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:51.559743 10336 rpcz_store.cc:269] 0709 12:11:29.065056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:29.065357 (+   301us) service_pool.cc:224] Handling call
0709 12:11:51.460544 (+22395187us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:46.014905 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:46.096104 (+ 81199us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:46.096151 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:11:46.096171 (+    20us) tablet.cc:662] Decoding operations
0709 12:11:46.097971 (+  1800us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:46.098039 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:46.098069 (+    30us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:11:46.331245 (+233176us) tablet.cc:701] Row locks acquired
0709 12:11:46.331272 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:11:46.331524 (+   252us) write_op.cc:270] Start()
0709 12:11:46.331680 (+   156us) write_op.cc:276] Timestamp: P: 1783599106331430 usec, L: 0
0709 12:11:46.331701 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:11:46.626373 (+294672us) log.cc:844] Serialized 7389 byte log entry
0709 12:11:47.327459 (+701086us) op_driver.cc:464] REPLICATION: finished
0709 12:11:47.511703 (+184244us) write_op.cc:301] APPLY: starting
0709 12:11:47.511805 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:47.949168 (+437363us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:47.949290 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:48.378384 (+429094us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b1800228460. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:11:48.625773 (+247389us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:48.626108 (+   335us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:11:48.626158 (+    50us) write_op.cc:312] APPLY: finished
0709 12:11:48.626595 (+   437us) log.cc:844] Serialized 172 byte log entry
0709 12:11:48.810635 (+184040us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:48.823329 (+ 12694us) write_op.cc:454] Released schema lock
0709 12:11:48.834830 (+ 11501us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:48.868051 (+ 33221us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:48.868113 (+    62us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":74568,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":183924,"apply.run_cpu_time_us":48692,"apply.run_wall_time_us":1356318,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":312875,"prepare.run_cpu_time_us":30812,"prepare.run_wall_time_us":823128,"raft.queue_time_us":176796,"raft.run_cpu_time_us":24283,"raft.run_wall_time_us":106395,"replication_time_us":995537,"spinlock_wait_cycles":12149888,"thread_start_us":825967,"threads_started":5,"wal-append.queue_time_us":345362,"wal-append.run_cpu_time_us":30289,"wal-append.run_wall_time_us":2526917}]]}
I20260709 12:11:51.567162 10585 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:11:51.488346 15548 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.494930 10522 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-cfb1df1b6ba3414899a137b410511b26 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:11:51.571557 15627 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.572018 15548 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.424602 11212 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:51.546017 10513 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:51.480041 10511 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:11:51.478386 15499 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.481105 14901 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.569461 15467 raft_consensus.cc:2764] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:11:51.572312 10522 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-cfb1df1b6ba3414899a137b410511b26 on tablet cfb1df1b6ba3414899a137b410511b26 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:51.495236 15615 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:51.594697 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 402851us
I20260709 12:11:51.599081 10585 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:51.580680 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.603137 15499 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:51.604736 15572 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:51.607707 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.629325 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:51.632185 10508 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.554s	user 0.014s	sys 0.000s
I20260709 12:11:51.632679 10508 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:51.605386 15572 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.526859 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:51.622853 10580 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.642628 10584 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:51.643304 10583 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:51.635691 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:51.651299 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.652091 11216 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:11:51.654479 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.655234 10578 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:51.638583 15582 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.666190 15582 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.668303 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.669106 10584 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.669471 15582 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.671640 15548 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:51.674119 15584 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:51.674753 15584 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:51.675814 15584 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.679039 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:51.603194 15615 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.679908 11220 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:51.680294 11218 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:51.665113 15590 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:51.681257 15590 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:51.683593 14901 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.653191 15519 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:51.684640 15519 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.685050 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:51.682649 15590 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.692817 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:51.690197 11218 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:51.658763 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:51.679241 15627 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:51.699335 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.700600 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.702348 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:51.705060 10579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.013s	sys 0.010s
W20260709 12:11:51.708211 10579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.013s	sys 0.010s
I20260709 12:11:51.681641 15615 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.686893 15519 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.720225 10584 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:51.722169 10583 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:11:51.718874 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:51.723491 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 196768us
I20260709 12:11:51.707537 15582 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:51.728850 10580 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.733355 10582 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:51.714690 15458 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:11:51.711159 15459 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:51.736338 10585 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:11:51.742095 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:51.761000 10584 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:51.720045 15334 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:51.731680 15582 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.734687 15548 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:51.764860 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104911us
I20260709 12:11:51.766531 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.769526 10583 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:11:51.776373 15334 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.783530 10582 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:51.784497 10552 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:51.787930 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 236934us
I20260709 12:11:51.788550 15648 raft_consensus.cc:2764] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:11:51.752240 15499 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:51.790618 15499 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.790328 15519 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:51.791052 15499 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:51.791344 15519 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:51.791339 15627 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.791914 15519 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:51.792325 15627 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.792950 15519 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.794373 15519 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:51.795233 15615 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:51.795809 15615 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:51.796824 15615 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.762148 15458 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:51.799614 15582 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.799996 15458 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.790328 15572 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.802345 15334 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:51.798847 10585 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:11:51.803539 15459 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:51.809002 15459 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.812580 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.815054 15552 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:51.815546 15552 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:51.815999 15552 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.819394 15548 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:51.827380 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.828626 15548 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:51.830154 15627 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:51.846043 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:51.868098 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 224924us
I20260709 12:11:51.895138 10578 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:11:51.871021 15407 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:51.901870 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55980.1us
W20260709 12:11:51.911020 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.879756 14901 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:11:51.908584 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 118540us
I20260709 12:11:51.932943 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:51.945041 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.926056 15548 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:11:51.942693 15334 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:51.947695 15334 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:51.942054 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:51.948400 14901 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:11:51.934073 15471 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:51.957844 15334 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.960539 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:51.964756 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 222748us
I20260709 12:11:51.968567 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.969149 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.969199 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:51.960842 15673 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:11:51.973448 10553 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:51.976471 10553 leader_election.cc:403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:51.977053 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.973748 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.979372 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:51.980259 10511 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:51.967173 15645 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:51.971861 15471 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:51.984551 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:51.988866 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:51.973686 15627 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:51.991195 14901 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:51.992481 15471 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:51.986004 15672 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:11:51.977433 15648 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
W20260709 12:11:52.039791 15681 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:11:52.041554 10586 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:11:52.049147 11215 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:52.049355 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:52.050308 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:52.052498 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:51.993254 15334 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.052896 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:52.053241 10511 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:52.053659 10511 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.054279 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:51.993336 15467 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:52.055243 15467 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:52.054610 10511 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.042083 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:52.063447 10511 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:52.043514 15678 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:11:52.041007 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56548.6us
I20260709 12:11:51.986590 15645 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.066211 15645 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:52.066671 15676 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:11:52.041884 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 62684.3us
I20260709 12:11:52.055675 15467 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.045711 15685 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:52.071676 15685 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:52.072299 15552 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.072829 15685 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.046346 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:52.059682 15680 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:11:52.056027 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:52.060091 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.061992 15584 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:52.060539 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:52.076647 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.099668 15648 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:52.100374 15648 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.100775 15648 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.048072 15590 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:52.101835 15590 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.102191 15590 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.084347 15584 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:52.104072 15584 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.105201 15584 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:52.107519 15552 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.108021 11214 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:11:52.082868 10511 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:52.116173 11177 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:52.116469 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.117161 15615 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:52.117583 15615 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:52.117955 15615 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.121622 10510 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:52.106509 15582 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.112959 15572 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:11:52.125882 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 164939us
W20260709 12:11:52.134773 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:52.129679 15549 consensus_meta.cc:287] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.568s	user 0.014s	sys 0.001s
W20260709 12:11:52.137413 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 204546us
I20260709 12:11:52.130163 15334 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:52.148975 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 108163us
W20260709 12:11:52.150470 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66149us
I20260709 12:11:52.150861 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:52.137125 15572 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:52.162130 10552 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:52.158389 15691 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:52.145839 15683 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:11:52.152159 15334 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.159487 15499 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.184365 15695 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:52.192669 15334 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:52.203316 15572 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.214030 15695 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:52.214251 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48068.8us
I20260709 12:11:52.215262 15695 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.216449 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.172984 15691 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.217519 15691 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.217885 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:52.224380 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.231844 15699 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:52.232750 15699 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:52.233481 15699 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.241824 15699 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:52.242014 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 91249.3us
W20260709 12:11:52.262583 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 100500us
I20260709 12:11:52.234061 15692 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.228566 15627 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:11:52.277153 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.280658 10513 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:11:52.262001 15687 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:52.288372 10511 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:52.307572 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:52.203606 15590 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:52.321678 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44656.4us
I20260709 12:11:52.322041 15582 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.322810 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.323365 15705 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:52.324062 15687 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.329205 15572 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:11:52.332653 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:52.336004 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.309262 15627 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:11:52.360818 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 197532us
I20260709 12:11:52.361740 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:52.365842 10587 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.369501 15699 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:52.369987 15699 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.371079 15699 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.372555 15699 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:52.315719 15519 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:11:52.364878 15690 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:11:52.386047 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 133128us
I20260709 12:11:52.387957 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:52.416296 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93598.3us
I20260709 12:11:52.421653 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.435403 10586 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19. Ops: []
I20260709 12:11:52.433874 15690 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:52.449466 15690 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.449942 15690 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:52.415263 15697 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:11:52.455616 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.457080 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.457967 15701 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:52.458406 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.458841 10587 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:52.460497 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:52.476719 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.475420 15548 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:52.477741 15471 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:52.479606 15548 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:52.477699 15499 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:52.493139 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.429806 15687 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:52.504205 15711 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:52.504511 15590 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:52.504860 15711 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.505302 15711 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.509693 15467 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.512671 15467 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.459227 15645 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.458411 15701 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.491900 15407 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:52.516871 10513 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:52.523953 15706 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:11:52.533360 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.499995 15582 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:52.474196 15459 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
W20260709 12:11:52.536624 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.480106 15548 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:52.538372 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45377.9us
W20260709 12:11:52.546459 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.475849 15519 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:52.562013 15519 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.562587 15645 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:52.539844 15711 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.600543 10331 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 111) took 6776 ms. Trace:
I20260709 12:11:52.563104 15519 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.496397 15615 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.550930 15701 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.567095 15698 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.609155 15615 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.609494 15698 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.567646 15695 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:52.568452 14901 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:52.607518 10331 rpcz_store.cc:276] 0709 12:11:45.840872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.843167 (+  2295us) service_pool.cc:224] Handling call
0709 12:11:52.600446 (+6757279us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:51.322416 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:51.323379 (+   963us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:51.323451 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:11:51.323503 (+    52us) tablet.cc:662] Decoding operations
0709 12:11:51.475493 (+151990us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:51.475596 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:51.475626 (+    30us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:11:51.476533 (+   907us) tablet.cc:701] Row locks acquired
0709 12:11:51.476586 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:11:51.476785 (+   199us) write_op.cc:270] Start()
0709 12:11:51.476955 (+   170us) write_op.cc:276] Timestamp: P: 1783599111476709 usec, L: 0
0709 12:11:51.477006 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 12:11:51.494537 (+ 17531us) log.cc:844] Serialized 3669 byte log entry
0709 12:11:51.914111 (+419574us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adae28 00007f2c96515581 00007f2c96514d58 00007f2c965199f1 00007f2c965199a9 00007f2c96519939 00007f2c96519901 00007f2c96518b2d 0000000000388634 0000000000388479
0709 12:11:52.131124 (+217013us) op_driver.cc:464] REPLICATION: finished
0709 12:11:52.176593 (+ 45469us) write_op.cc:301] APPLY: starting
0709 12:11:52.206004 (+ 29411us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:52.507974 (+301970us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:52.508026 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:52.572463 (+ 64437us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:52.572772 (+   309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:11:52.572835 (+    63us) write_op.cc:312] APPLY: finished
0709 12:11:52.573234 (+   399us) log.cc:844] Serialized 89 byte log entry
0709 12:11:52.573481 (+   247us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:52.573635 (+   154us) write_op.cc:454] Released schema lock
0709 12:11:52.573713 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:52.574220 (+   507us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:52.574251 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":432207,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":33760,"apply.run_cpu_time_us":18099,"apply.run_wall_time_us":404621,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41765,"num_ops":9,"prepare.queue_time_us":191908,"prepare.run_cpu_time_us":107464,"prepare.run_wall_time_us":620817,"raft.queue_time_us":137903,"raft.run_cpu_time_us":1906,"raft.run_wall_time_us":1906,"replication_time_us":653917,"spinlock_wait_cycles":38321024,"thread_start_us":570709,"threads_started":4,"wal-append.queue_time_us":427840}]]}
I20260709 12:11:52.546133 15471 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:52.613332 15471 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.613824 15471 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:52.615061 15698 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:52.615168 15499 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.615832 15499 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.491058 15708 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 12:11:52.577780 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:52.617900 11215 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.618251 15708 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:52.546311 15549 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.620035 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:52.620282 15549 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.620740 10586 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.622552 15549 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.578573 15459 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:11:52.623168 15459 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:52.623369 15645 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:11:52.624224 15459 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.624738 15645 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:52.625221 15645 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:52.625610 15459 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:11:52.627826 15499 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.628072 15471 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:52.587893 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.631835 15471 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.641672 10333 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 9, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 9 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:52.605307 15692 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:52.607075 15519 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:52.603700 15648 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.003s
I20260709 12:11:52.612818 14901 raft_consensus.cc:3055] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:52.646422 14901 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:52.645859 10333 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 10 to 13, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 13 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:52.647881 15458 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:52.648545 15648 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.638046 11214 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:11:52.535634 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 118695us
I20260709 12:11:52.559269 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:52.654105 15692 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:52.654659 15692 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:52.647604 14901 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.648514 15458 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.656065 14901 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:52.657226 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 98115.6us
I20260709 12:11:52.657886 15458 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:52.586517 15582 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:52.652303 10333 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 8 to 10, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 10 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:52.653105 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:52.659691 10438 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:52.659950 15582 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.660842 15590 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:52.661490 10437 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 12:11:52.661180 15712 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:52.662498 15590 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.660490 10333 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 13, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 13 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:52.664717 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:52.618705 15708 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.601945 15711 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.655691 15692 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:52.643898 15584 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.002s
I20260709 12:11:52.577517 15691 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.668311 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:52.672644 10436 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:11:52.673702 10433 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:11:52.674269 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:52.674793 10429 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:11:52.675585 10508 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:52.675717 10435 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:52.663021 15683 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:52.680635 10586 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
W20260709 12:11:52.683117 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.684082 10432 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:52.676586 10333 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 9, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 9 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:11:52.691020 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 303099us
I20260709 12:11:52.692371 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:52.697386 11217 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:52.697881 10507 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:52.699003 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.687677 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.692617 10587 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:11:52.698249 10507 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.695680 10429 pending_rounds.cc:85] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:11:52.703154 11214 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:11:52.703867 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.704262 10438 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:52.704583 10432 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:52.705119 10437 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:11:52.707350 10431 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 11. Ops: []
I20260709 12:11:52.708016 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:52.708801 11220 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:52.720605 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:52.667595 15695 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.722319 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.721684 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:52.723012 10434 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:52.703306 10429 pending_rounds.cc:107] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:11:52.731950 10429 raft_consensus.cc:2889] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:52.732357 10429 pending_rounds.cc:107] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:11:52.734208 10429 raft_consensus.cc:2889] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:52.736047 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.736873 10435 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:52.725020 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:52.700793 15683 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.742010 11215 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:52.745600 11177 leader_election.cc:400] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:11:52.747784 10433 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:52.756844 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:52.735247 15458 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:11:52.679870 15698 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:11:52.717978 15711 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:52.766721 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.766431 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:52.772074 10437 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:52.766722 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.699103 15615 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:11:52.773018 10434 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:52.766417 15715 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:11:52.775429 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:52.771942 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:52.784690 10438 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.775748 10435 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:52.767459 15549 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:52.776321 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.788223 10429 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
W20260709 12:11:52.779702 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.781451 15691 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.780858 10333 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 6 to 8, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 8 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:11:52.784098 11215 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.775604 10432 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:52.796236 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:52.797029 10436 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.763188 15699 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:11:52.759287 10507 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.802578 10507 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.731096 15690 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.813149 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:52.814003 10437 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.736178 15695 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.795795 15407 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:52.838255 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.801580 15549 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:52.773293 15685 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:11:52.839624 10433 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:52.783134 15711 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.782832 15710 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:11:52.735567 15590 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.843662 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:52.746562 15645 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.842819 10435 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:52.818449 15648 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:52.843740 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:52.867740 10508 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:52.867894 10507 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:11:52.850603 11215 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:52.851104 15724 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:52.819923 15467 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:11:52.852316 15458 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:11:52.860224 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50182.9us
I20260709 12:11:52.868306 10507 pending_rounds.cc:85] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 11
I20260709 12:11:52.888301 10507 pending_rounds.cc:107] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:11:52.888705 10507 raft_consensus.cc:2889] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:52.877017 11213 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
W20260709 12:11:52.882174 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:52.837282 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76004.7us
I20260709 12:11:52.910974 11177 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:52.912966 15722 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:52.913285 15615 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.913599 15722 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:52.913667 10586 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:52.913884 15615 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.914870 15722 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.916621 15615 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:52.917383 15615 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:52.918886 15698 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.919482 15698 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.922314 15698 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:52.890877 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.900229 15467 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:52.901000 15689 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:52.805135 15708 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.925328 15713 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:52.925895 15708 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.927222 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:52.869247 15724 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:11:52.931180 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.938424 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.947707 10434 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:52.963791 10508 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:52.906574 15723 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:52.881356 15548 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.979156 11220 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:11:52.984052 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.985898 15519 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.989180 15519 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.991135 15713 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:52.991575 15713 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:52.992501 15713 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:52.993512 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:52.999715 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53412.3us
I20260709 12:11:52.865377 15683 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:11:53.001720 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 102187us
I20260709 12:11:53.017714 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:53.022706 10433 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:53.030848 11216 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
W20260709 12:11:53.034169 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 170658us
I20260709 12:11:53.037472 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:53.037846 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.021291 15467 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.040581 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.980093 15727 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:11:53.022862 15713 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.043394 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.044570 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:52.973404 15407 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:52.986047 15549 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.048444 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.044101 10429 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:53.057317 10505 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:53.066243 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:53.067030 10579 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:11:53.068595 15719 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:11:53.023450 15717 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:11:53.042439 15698 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.045459 15519 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:53.100062 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:53.103209 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:52.986016 15723 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.044744 15690 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:53.111806 10433 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:11:53.126607 10585 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:11:53.135546 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.065219 15695 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.140558 15548 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:53.147542 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:53.152504 15459 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 12:11:53.051817 15467 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.155308 15467 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:53.159502 11212 raft_consensus.cc:2410] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:11:53.150602 15727 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
W20260709 12:11:52.931129 15687 consensus_queue.cc:1175] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 11 status { last_received { term: 10 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:11:53.167268 15467 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:53.168047 15467 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.169128 15467 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.108559 15711 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:53.182173 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.183028 10583 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:53.125631 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:53.181249 15701 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.573s	user 0.000s	sys 0.013s
I20260709 12:11:53.194732 15584 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:53.195351 15584 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:53.205610 10438 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:53.229905 15584 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.231691 15584 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:53.233460 15648 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:53.234149 15648 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:53.241922 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.243788 15683 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:53.248378 15716 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.399s	user 0.000s	sys 0.001s
I20260709 12:11:53.250717 15645 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:53.252091 15728 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.390s	user 0.002s	sys 0.000s
W20260709 12:11:53.256143 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 210161us
I20260709 12:11:53.257244 10436 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:53.257496 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:53.259968 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:53.265609 10429 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:11:53.270308 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:53.273435 15694 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.188934 15701 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.278136 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.115195 15698 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.303762 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.309870 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.274127 15694 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.311452  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:53.311710 11178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.016s	sys 0.103s
W20260709 12:11:53.312193 11178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.016s	sys 0.103s
I20260709 12:11:53.313845 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.314019 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.314868 10509 raft_consensus.cc:2410] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:11:53.315176 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:53.318153 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 137849us
I20260709 12:11:53.319041 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:53.214709 10579 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:11:53.323247 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:53.323421 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:53.324156 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.328042 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.329066 10431 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:53.332065 10509 raft_consensus.cc:2368] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:11:53.333046 10430 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:53.339426 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:53.303793 15689 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:53.327881 15695 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:53.350754 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.354902 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.350775 14901 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:53.354251 10478 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:53.365410 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.363692 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:53.372324 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.373051 15704 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:53.373153 14901 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.373767 15704 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:53.374197 15704 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.375435 15729 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:53.375679 14901 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.376116 15729 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.377573 15695 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:53.378211 15711 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:53.378746 15711 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:53.379256 15711 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:53.378895 15695 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.331939 15715 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:11:53.384343 15740 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.384974 15740 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.366556 10504 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:11:53.402084 10507 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:53.416296 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.417222 15697 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:53.417994 10582 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:53.417397 10583 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:53.420480 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.433771 10436 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:53.436035 10579 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:11:53.439739 10429 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:53.196362 15724 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.350749 15590 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
W20260709 12:11:53.474454 15735 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:11:53.479063 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:53.506312 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:53.519869 15714 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.478s	user 0.004s	sys 0.000s
I20260709 12:11:53.521898 10511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2505) took 1187 ms. Trace:
I20260709 12:11:53.522423 10511 rpcz_store.cc:276] 0709 12:11:52.334524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.334827 (+   303us) service_pool.cc:224] Handling call
0709 12:11:52.335278 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:52.366959 (+ 31681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:52.367036 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:52.385661 (+ 18625us) log.cc:844] Serialized 26 byte log entry
0709 12:11:52.546554 (+160893us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:52.546605 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:52.546692 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:53.183782 (+637090us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:53.521449 (+337667us) raft_consensus.cc:1685] finished
0709 12:11:53.521479 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:53.521786 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26249,"raft.queue_time_us":67296,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":12197,"spinlock_wait_cycles":93187840,"thread_start_us":115176,"threads_started":1,"wal-append.queue_time_us":321462}
I20260709 12:11:53.447190 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.534247 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.535437 10507 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:53.544275 10437 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.707s	user 0.019s	sys 0.002s
I20260709 12:11:53.545379 10437 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:11:53.586709 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:53.546342 15699 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.006s	sys 0.014s
I20260709 12:11:53.511507 15697 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:53.601661 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.546103 15467 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.619380 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.620580 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.623471 15723 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.606880 15724 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:53.631165 15685 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:53.631691 15740 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:53.637463 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.660311 11178 leader_election.cc:400] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:11:53.637790 15732 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:11:53.607365 15699 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.007s	sys 0.015s
W20260709 12:11:53.657394 10403 connection.cc:256] RPC call timeout handler was delayed by -2.4300664680001773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
I20260709 12:11:53.660653 11178 leader_election.cc:403] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:11:53.615552 15334 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.671036 10437 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:53.674002 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.672394 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.671725 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.648180 15687 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.675366 10431 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:53.677523 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.678956 10429 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:53.680132 10433 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:53.681185 10438 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:53.694065 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.697070 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.697537 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.697880 10430 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:53.677819 15677 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:11:53.707142 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.710615 10436 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:53.714576 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.715445 10437 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:53.716632 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.736142 10435 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:53.678334 15727 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.647115 15697 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:53.740304 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.741528 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:53.741878 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:53.674861 15699 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:53.745909 10584 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.019s	sys 0.018s
I20260709 12:11:53.748111 10431 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:53.693292 15723 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:11:53.748523 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:53.752904 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:53.748106 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:53.752948 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.753221 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:53.755205 10584 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.019s	sys 0.018s
I20260709 12:11:53.755672 10429 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:11:53.701586 15701 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:53.713685 15705 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:53.755256 10433 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:53.696507 15694 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.759456 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:53.762928 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.763674 10431 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:53.762446 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:53.762728 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.764158 10436 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:53.764269 10434 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:53.766100 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.682611 15334 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.772293 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.772970 10435 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:53.770079 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.744167 15648 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.779165 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.779881 10579 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
W20260709 12:11:53.769140 10585 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.061s
I20260709 12:11:53.764472 10430 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
W20260709 12:11:53.770955 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.773890 10431 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:11:53.777827 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:53.783012 10585 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.009s	sys 0.061s
I20260709 12:11:53.702911 15687 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.787407 15734 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:53.787917 15734 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.788264 15734 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:53.789511 15687 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:53.790155 15687 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:53.790863 15687 consensus_queue.cc:1243] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:53.791221 15519 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:11:53.793552 15519 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:53.793911 15519 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.795297 15712 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:53.795711 15712 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.796567 10434 raft_consensus.cc:2410] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:11:53.796759 15712 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.797479 15334 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.799240 15712 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:53.799722 10479 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:53.801518 15748 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:53.802129 15748 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:53.802709 15748 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.802939 15709 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:53.804703 15709 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:53.801862 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:53.802268 15687 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:53.791450 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:53.807607 15699 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.808885 10579 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:53.810104 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:53.757411 15740 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 12:11:53.821880 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.794818 15519 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.753942 15727 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.824748 15697 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:53.833987 10507 raft_consensus.cc:2393] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:53.834314 15727 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:53.760944 15705 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.835809 15705 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:53.836933 15705 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:53.825234 15615 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:53.837734 15615 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:53.839558 11212 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:53.782963 15590 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:11:53.828099 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.841881 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.810853 10579 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:53.844808 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.850713 10433 raft_consensus.cc:2410] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:11:53.851915 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.860723 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.861726 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:53.824147 15519 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:53.865767 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:53.832288 15741 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:53.875300 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.886147 10433 raft_consensus.cc:2410] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:11:53.850003 15742 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:53.884105 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:53.891206 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:53.894941 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.832324 15731 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 12:11:53.870977 15689 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:53.874198 15700 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:53.889827 15645 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.002s
I20260709 12:11:53.860276 15717 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:11:53.864714 15739 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:11:53.914644 10433 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:53.861982 15690 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:11:53.872782 15740 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:11:53.935658 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.889995 15742 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:53.927327 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.838887 15615 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:53.930888 10579 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:53.903868 15709 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:11:53.939755 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:53.939378 11177 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.000s	sys 0.015s
W20260709 12:11:54.012724 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 257988us
W20260709 12:11:54.013204 11177 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.000s	sys 0.016s
W20260709 12:11:54.013506 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:54.013682 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 589228us
I20260709 12:11:53.897243 15741 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.942020 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.031812 10433 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:53.952701 10432 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:53.952747 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:53.915362 15685 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.957585 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:53.931425 15549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.012s	sys 0.002s
I20260709 12:11:53.928129 15689 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.985484 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.987083 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:53.986886 10436 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:53.898175 15590 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
W20260709 12:11:53.990018 10587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.014s	sys 0.009s
W20260709 12:11:54.075373 10587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.015s	sys 0.009s
I20260709 12:11:53.931354 15700 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:53.993012 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:53.956193 15584 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.724s	user 0.015s	sys 0.000s
I20260709 12:11:53.950048 15690 consensus_queue.cc:1243] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:11:54.003788 15711 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.624s	user 0.023s	sys 0.004s
I20260709 12:11:53.948464 15742 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:53.997308 15708 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.071s	user 0.011s	sys 0.006s
W20260709 12:11:53.997097 15683 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.753s	user 0.018s	sys 0.009s
I20260709 12:11:54.032105 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.099968 10513 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:54.100993 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.102221 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.103332 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.104089 10579 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:11:54.104905 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.106087 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.035996 10438 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:53.994138 15615 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:54.107190 10504 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:11:54.107939 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.110420 11220 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:54.083608 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2890) took 1033 ms. Trace:
I20260709 12:11:54.085836 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.086591 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:54.072759 10429 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:54.044204 15549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.013s	sys 0.002s
I20260709 12:11:54.089732 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.120469 10435 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:54.084360 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:54.030463 15645 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:54.098632 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:54.106163 15704 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.014s
W20260709 12:11:54.117517 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 226575us
I20260709 12:11:54.119539 15708 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.111258 10587 rpcz_store.cc:276] 0709 12:11:53.050539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.051499 (+   960us) service_pool.cc:224] Handling call
0709 12:11:53.051816 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:53.052070 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:53.444626 (+392556us) log.cc:844] Serialized 14 byte log entry
0709 12:11:54.083101 (+638475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:54.083183 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:54.083252 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.083352 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.083524 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7110,"prepare.queue_time_us":574105,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":8324,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":1005028,"threads_started":2}
I20260709 12:11:54.121207 10581 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
W20260709 12:11:54.097506 15713 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.047s	user 0.015s	sys 0.010s
I20260709 12:11:54.234297 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.134941 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:54.237782 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 155060us
W20260709 12:11:54.238755 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.239032 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:54.081034 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.240657 10436 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:11:54.241454 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:54.243120 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.245448 10429 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:54.112445 15584 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:54.247282 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 108825us
I20260709 12:11:54.263900 10510 raft_consensus.cc:2410] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:11:54.073088 15685 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.142174 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:54.253814 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.112578 15731 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.291538 15731 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.293988 15731 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.143653 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.295452 10580 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:54.297015 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.255012 10579 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:54.299580 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.300628 10587 raft_consensus.cc:2376] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14. Re-sending same reply.
I20260709 12:11:54.134075 15683 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:54.096922 15711 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.133769 15741 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.435030 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:54.370769 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.363404 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:54.331851 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.364746 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.440836 10429 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:54.441087 10430 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:54.364235 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:54.420078 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:54.349800 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 207919us
W20260709 12:11:54.350101 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 111230us
W20260709 12:11:54.348129 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91631.4us
I20260709 12:11:54.365933 10438 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:54.334092 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.443040 10580 raft_consensus.cc:2410] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:11:54.330910 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.443377 10434 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:54.279368 15749 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:11:54.429327 10507 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
W20260709 12:11:54.432228 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.450699 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:54.443080 11177 leader_election.cc:400] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:11:54.363878 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.446434 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.447666 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.456368 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 2520) took 1403 ms. Trace:
I20260709 12:11:54.457759 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2892) took 1384 ms. Trace:
W20260709 12:11:54.333858 15754 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
W20260709 12:11:54.340185 15756 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
W20260709 12:11:54.346279 15760 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 12:11:54.333123 15763 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:11:54.283690 15698 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:54.496937 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.497748 10431 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:54.332787 15750 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 12:11:54.464573 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.505555 10437 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:54.505759 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.302829 15584 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.456874 10513 rpcz_store.cc:276] 0709 12:11:53.036750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.037154 (+   404us) service_pool.cc:224] Handling call
0709 12:11:54.439999 (+1402845us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400152054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:54.440119 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441823616}
W20260709 12:11:54.360776 15712 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.561s	user 0.007s	sys 0.007s
I20260709 12:11:54.472714 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.303261 15766 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:54.475711 14901 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.308337 15683 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:54.256685 15572 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.197s
I20260709 12:11:54.143149 15704 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.248155 15645 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.245491 15708 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.481716 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.497423 15713 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:54.532470 15687 consensus_meta.cc:287] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.726s	user 0.013s	sys 0.000s
I20260709 12:11:54.345429 15732 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:54.488673 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.490141 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.491176 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 566) took 1218 ms. Trace:
I20260709 12:11:54.496220 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.343360 15767 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:54.302925 15729 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:54.501879 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45788.3us
I20260709 12:11:54.581248 10510 rpcz_store.cc:276] 0709 12:11:53.272783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.272915 (+   132us) service_pool.cc:224] Handling call
0709 12:11:54.490560 (+1217645us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54002f2bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:54.491099 (+   539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204616064}
I20260709 12:11:54.585767 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:54.502255 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.502600 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 570) took 1225 ms. Trace:
I20260709 12:11:54.432578 15690 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:54.457067 15701 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.458190 10581 rpcz_store.cc:276] 0709 12:11:53.054270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.054434 (+   164us) service_pool.cc:224] Handling call
0709 12:11:54.437380 (+1382946us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b540010c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:54.437486 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1879237888}
I20260709 12:11:54.519436 10506 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19. Ops: []
I20260709 12:11:54.522248 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.519598 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.524008 14901 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.333647 15700 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:54.613579 15773 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 12:11:54.489105 15748 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.686s	user 0.013s	sys 0.001s
I20260709 12:11:54.615317 15748 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.526808 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.616699 10587 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:54.618060 15748 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.478653 15774 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.619379 15774 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:54.620520 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.621567 10509 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:11:54.622004 10509 pending_rounds.cc:85] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 10
I20260709 12:11:54.622409 10509 pending_rounds.cc:107] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:11:54.622735 10509 raft_consensus.cc:2889] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:54.533968 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.624745 10583 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:54.624954 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 676) took 1302 ms. Trace:
I20260709 12:11:54.625707 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 575) took 1346 ms. Trace:
I20260709 12:11:54.546813 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:54.627326 15734 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.839s	user 0.013s	sys 0.003s
I20260709 12:11:54.627622 10507 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:54.478502 15731 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.627825 15734 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.628243 15731 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.350068 15717 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:54.556301 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2896) took 1437 ms. Trace:
I20260709 12:11:54.625465 11215 rpcz_store.cc:276] 0709 12:11:53.322700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.332304 (+  9604us) service_pool.cc:224] Handling call
0709 12:11:53.354094 (+ 21790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.354488 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.354518 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.391276 (+ 36758us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.780159 (+388883us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:53.780203 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.780285 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.624673 (+844388us) raft_consensus.cc:1685] finished
0709 12:11:54.624703 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.624857 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9880,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":26954,"thread_start_us":308873,"threads_started":1,"wal-append.queue_time_us":513130}
I20260709 12:11:54.637151 10584 rpcz_store.cc:276] 0709 12:11:53.118767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.118902 (+   135us) service_pool.cc:224] Handling call
0709 12:11:53.125684 (+  6782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.230796 (+105112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.230831 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.231480 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.755884 (+524404us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:53.755943 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.756088 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.556081 (+799993us) raft_consensus.cc:1685] finished
0709 12:11:54.556105 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.556233 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76250,"raft.queue_time_us":114404,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":131,"spinlock_wait_cycles":43392,"thread_start_us":523927,"threads_started":1,"wal-append.queue_time_us":665083}
I20260709 12:11:54.556630 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.579823 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.622051 15774 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:54.584448 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:54.587559 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.587548 10508 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:54.556329 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:54.586757 10512 rpcz_store.cc:276] 0709 12:11:53.277642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.277792 (+   150us) service_pool.cc:224] Handling call
0709 12:11:54.502347 (+1224555us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002c65d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:54.502525 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2669414144}
W20260709 12:11:54.487912 15467 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.822s	user 0.013s	sys 0.000s
I20260709 12:11:54.654289 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.598536 15727 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:11:54.603860 15775 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:11:54.604823 15690 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.672577 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.673395 10504 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:54.673954 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2652) took 1003 ms. Trace:
I20260709 12:11:54.674327 10504 rpcz_store.cc:276] 0709 12:11:53.670628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.645251 (+974623us) service_pool.cc:224] Handling call
0709 12:11:54.673882 (+ 28631us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.607326 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 572) took 1329 ms. Trace:
W20260709 12:11:54.608034 11177 leader_election.cc:336] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.773s (SENT)
I20260709 12:11:54.339246 15689 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.609215 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.676013 10581 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:11:54.609760 15765 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.276s	user 0.003s	sys 0.000s
I20260709 12:11:54.610131 15704 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:54.610400 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2905) took 1195 ms. Trace:
I20260709 12:11:54.704600 15731 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.704793 10582 rpcz_store.cc:276] 0709 12:11:53.415410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.448615 (+ 33205us) service_pool.cc:224] Handling call
0709 12:11:53.476464 (+ 27849us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.512865 (+ 36401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.512920 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.588691 (+ 75771us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.897526 (+308835us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:53.897573 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.897653 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.610109 (+712456us) raft_consensus.cc:1685] finished
0709 12:11:54.610143 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.610320 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1696,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":49131,"thread_start_us":271367,"threads_started":1,"wal-append.queue_time_us":404592}
I20260709 12:11:54.707139 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.611886 15709 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:54.612105 15645 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:54.709085 15709 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:54.710173 15709 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.711398 15709 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:54.715797 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.716549 10504 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:54.717294 15708 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.717527 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.636469 15701 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.718366 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2633) took 1107 ms. Trace:
I20260709 12:11:54.618716 15739 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:54.718940 10432 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:54.719233 15739 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:54.719866 15685 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.718780 10504 rpcz_store.cc:276] 0709 12:11:53.611210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.717247 (+1106037us) service_pool.cc:224] Handling call
0709 12:11:54.718308 (+  1061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.720944 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:54.720824 15739 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.721665 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.724282 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.679831 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:54.629909 10511 raft_consensus.cc:2368] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:54.626259 10509 rpcz_store.cc:276] 0709 12:11:53.279375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.372576 (+ 93201us) service_pool.cc:224] Handling call
0709 12:11:54.490765 (+1118189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.623303 (+132538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.625007 (+  1704us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54002f2bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:54.625508 (+   501us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002f2bd0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:54.625646 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2870538880}
I20260709 12:11:54.581323 15712 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.535194 15407 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:54.640255 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2898) took 1498 ms. Trace:
I20260709 12:11:54.641333 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2893) took 1570 ms. Trace:
I20260709 12:11:54.581782 15687 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:54.643215 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.645651 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.647629 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2637) took 1035 ms. Trace:
I20260709 12:11:54.649713 10434 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:54.594893 15766 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:54.658865 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:54.659920 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.749395 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.666894 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 901) took 1398 ms. Trace:
I20260709 12:11:54.750264 10435 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:54.559125 15713 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.683107 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:54.754070 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 71080.4us
I20260709 12:11:54.755296 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:54.744993 15407 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:54.758662 15584 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:54.759392 15584 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:54.759049 15407 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.759923 15584 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.760367 15584 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:54.760723 15407 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:54.744923 15704 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:54.762781 15782 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:11:54.763459 15761 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:54.675154 10506 rpcz_store.cc:276] 0709 12:11:53.278327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.278449 (+   122us) service_pool.cc:224] Handling call
0709 12:11:54.519187 (+1240738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.605736 (+ 86549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.607121 (+  1385us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002c65d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:54.607257 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724259712}
I20260709 12:11:54.703819 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.615983 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:54.747012 10507 rpcz_store.cc:276] 0709 12:11:53.612387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.546417 (+934030us) service_pool.cc:224] Handling call
0709 12:11:54.647536 (+101119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.765369 10437 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:11:54.765460 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.728190 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:54.729022 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 121015us
I20260709 12:11:54.750602 10578 rpcz_store.cc:276] 0709 12:11:53.268895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.269752 (+   857us) service_pool.cc:224] Handling call
0709 12:11:53.304104 (+ 34352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.703689 (+399585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.703727 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.754382 (+ 50655us) log.cc:844] Serialized 26 byte log entry
0709 12:11:54.030967 (+276585us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:54.031046 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.031124 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.612864 (+581740us) spinlock_profiling.cc:243] Waited 94.6 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:54.657058 (+ 44194us) raft_consensus.cc:1685] finished
0709 12:11:54.657096 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.666799 (+  9703us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32167,"raft.queue_time_us":43132,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":2570,"spinlock_wait_cycles":217477888,"thread_start_us":542663,"threads_started":2,"wal-append.queue_time_us":434794}
I20260709 12:11:54.728302 10433 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:54.739635 10585 rpcz_store.cc:276] 0709 12:11:53.141806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.141991 (+   185us) service_pool.cc:224] Handling call
0709 12:11:53.147714 (+  5723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.253140 (+105426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.253175 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.253751 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.786589 (+532838us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:53.786654 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.786763 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.639936 (+853173us) raft_consensus.cc:1685] finished
0709 12:11:54.639966 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.640139 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150266,"raft.queue_time_us":105993,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":8910,"thread_start_us":532332,"threads_started":1,"wal-append.queue_time_us":650586}
I20260709 12:11:54.741199 10586 rpcz_store.cc:276] 0709 12:11:53.101971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.116608 (+ 14637us) service_pool.cc:224] Handling call
0709 12:11:53.117099 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.295753 (+178654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.295794 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.309370 (+ 13576us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.755711 (+446341us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:53.755757 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.755824 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.641056 (+885232us) raft_consensus.cc:1685] finished
0709 12:11:54.641090 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.641260 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137888,"raft.queue_time_us":140337,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":22462,"thread_start_us":401385,"threads_started":1,"wal-append.queue_time_us":538527}
I20260709 12:11:54.737419 14901 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:54.768059 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.768563 14901 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:54.769032 14901 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.595758 15732 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.748404 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:54.753046 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2639) took 1132 ms. Trace:
I20260709 12:11:54.774973 10510 rpcz_store.cc:276] 0709 12:11:53.620440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.586375 (+965935us) service_pool.cc:224] Handling call
0709 12:11:54.752971 (+166596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.752765 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2648) took 1129 ms. Trace:
I20260709 12:11:54.776552 10512 rpcz_store.cc:276] 0709 12:11:53.623227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.658587 (+1035360us) service_pool.cc:224] Handling call
0709 12:11:54.752684 (+ 94097us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.777822 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.762944 15704 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.750077 10513 raft_consensus.cc:2376] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14. Re-sending same reply.
I20260709 12:11:54.780941 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2635) took 1169 ms. Trace:
I20260709 12:11:54.696420 10587 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:54.781405 10513 rpcz_store.cc:276] 0709 12:11:53.611797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.659673 (+1047876us) service_pool.cc:224] Handling call
0709 12:11:54.780857 (+121184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.700287 15734 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.766273 10504 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:54.782882 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.783425 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2643) took 1161 ms. Trace:
I20260709 12:11:54.783885 10504 rpcz_store.cc:276] 0709 12:11:53.621708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.735759 (+1114051us) service_pool.cc:224] Handling call
0709 12:11:54.783343 (+ 47584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.636248 15549 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.729265 10505 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:11:54.789237 10587 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:54.658903 15727 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:54.769469 14901 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:54.746788 10580 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:54.770970 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2636) took 1158 ms. Trace:
I20260709 12:11:54.823781 10509 rpcz_store.cc:276] 0709 12:11:53.612097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.727862 (+1115765us) service_pool.cc:224] Handling call
0709 12:11:54.770882 (+ 43020us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.772336 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2910) took 1084 ms. Trace:
I20260709 12:11:54.774538 15701 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.786115 10584 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:54.640892 15683 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.799737 10579 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
W20260709 12:11:54.781606 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 209602us
I20260709 12:11:54.725083 15690 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:54.805675 10435 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:54.817185 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.836658 10507 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:54.822052 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2911) took 1133 ms. Trace:
I20260709 12:11:54.837500 10578 rpcz_store.cc:276] 0709 12:11:53.688543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.774152 (+1085609us) service_pool.cc:224] Handling call
0709 12:11:54.821346 (+ 47194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.821566 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.821591 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:54.821769 (+   178us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:54.821805 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.821862 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.821988 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:54.818017 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 164075us
I20260709 12:11:54.752779 15766 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:54.825661 10581 rpcz_store.cc:276] 0709 12:11:53.688230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.771345 (+1083115us) service_pool.cc:224] Handling call
0709 12:11:54.771737 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.771983 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.772021 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:54.772069 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:54.772102 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.772162 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.772269 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.827876 10513 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:54.828830 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.829571 10585 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:54.843269 15766 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:54.842931 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:54.844638 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2912) took 1155 ms. Trace:
I20260709 12:11:54.847455 10585 rpcz_store.cc:276] 0709 12:11:53.688850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.776359 (+1087509us) service_pool.cc:224] Handling call
0709 12:11:54.829290 (+ 52931us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.844347 (+ 15057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.844580 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.844564 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2524) took 1430 ms. Trace:
I20260709 12:11:54.848733 10513 rpcz_store.cc:276] 0709 12:11:53.413860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.827216 (+1413356us) service_pool.cc:224] Handling call
0709 12:11:54.827609 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.844186 (+ 16577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.844480 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.830562 10580 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:11:54.764039 15761 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:54.791489 15519 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.908s	user 0.015s	sys 0.001s
I20260709 12:11:54.836830 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.852469 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2634) took 1241 ms. Trace:
I20260709 12:11:54.852880 10504 rpcz_store.cc:276] 0709 12:11:53.611504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.784698 (+1173194us) service_pool.cc:224] Handling call
0709 12:11:54.852395 (+ 67697us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:54.856487 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:54.857366 10510 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:54.851502 10580 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:54.744799 15712 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.875517 10552 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:54.872812 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:54.901113 10580 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:11:54.901636 10580 pending_rounds.cc:85] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 7
I20260709 12:11:54.901876 10580 pending_rounds.cc:107] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:11:54.819896 15711 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:54.902139 10580 raft_consensus.cc:2889] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:54.917212 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:54.918220 10578 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:11:54.866077 15683 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:54.817878 15549 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:54.921659 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.969883 14901 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:54.895092 15685 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.894167 15572 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:54.993503 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:11:54.994240 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:54.995671 10429 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.002s
W20260709 12:11:54.997920 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 343151us
I20260709 12:11:54.907327 15762 raft_consensus.cc:2764] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 12:11:54.999066 10438 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.018s	sys 0.026s
W20260709 12:11:55.043282 10438 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.019s	sys 0.026s
I20260709 12:11:54.920159 15786 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:55.005513 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:55.006597 10429 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:55.018371 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:55.029884 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.040426 14901 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:55.039469 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:55.056805 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:55.057397 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58650.6us
I20260709 12:11:55.081792 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:55.088760 10583 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
W20260709 12:11:55.054402 15742 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.931s	user 0.012s	sys 0.003s
I20260709 12:11:55.096705 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:55.004640 15572 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.851064 15761 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:54.993916 15685 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:55.104418 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 232908us
W20260709 12:11:55.104772 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.059501 15789 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.081153 15711 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:55.109175 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:55.110144 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.115432 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 240049us
I20260709 12:11:54.913400 15727 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.117924 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2908) took 1430 ms. Trace:
I20260709 12:11:55.129112 15762 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:11:55.101155 15700 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.080379 15734 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:11:55.135548 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:55.186241 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50801.2us
I20260709 12:11:55.149333 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:55.151775 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0422285699999065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29900000000000004s for a response
I20260709 12:11:55.160145 10578 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:11:55.116986 15459 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:11:55.146343 15690 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.169688 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:55.203792 10583 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:55.203691 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:55.164043 10582 rpcz_store.cc:276] 0709 12:11:53.687538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.708102 (+1020564us) service_pool.cc:224] Handling call
0709 12:11:54.708433 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.708679 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:54.748767 (+ 40088us) log.cc:844] Serialized 14 byte log entry
0709 12:11:55.117339 (+368572us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:55.117501 (+   162us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:55.117570 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.117667 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.117839 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24977,"prepare.queue_time_us":44813,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":26947,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":363713,"threads_started":1}
I20260709 12:11:54.912796 15767 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
W20260709 12:11:55.201124 10553 leader_election.cc:336] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.599s (SENT)
I20260709 12:11:55.095408 15741 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:11:55.205952 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.204886 11215 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:11:55.207542 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 150867us
I20260709 12:11:55.149202 15713 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.209681 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:55.158424 15711 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.215684 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:55.144714 15742 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:55.184834 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 191426us
W20260709 12:11:55.216599 15791 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:11:55.203455 15774 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:55.200294 15700 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.219233 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.204517 15732 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.200017 15704 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.225896 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:55.203256 15459 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.205506 15690 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.215303 15727 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:55.167043 15789 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.211464 15698 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:55.240795 15794 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:11:55.209067 15783 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:55.242254 15742 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.242530 15783 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.243621 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:55.247196 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.222810 15772 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.356s	user 0.002s	sys 0.000s
I20260709 12:11:55.236940 10512 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:55.218590 15711 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:55.247022 15788 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.260836 15788 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.262544 15788 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:55.221541 15712 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.250676 15690 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:55.265339 15767 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:55.267890 11219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 675) took 1946 ms. Trace:
I20260709 12:11:55.268366 11219 rpcz_store.cc:276] 0709 12:11:53.321276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.321469 (+   193us) service_pool.cc:224] Handling call
0709 12:11:53.321949 (+   480us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:53.337058 (+ 15109us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:11:53.763506 (+426448us) log.cc:844] Serialized 14 byte log entry
0709 12:11:54.074449 (+310943us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:54.116854 (+ 42405us) log.cc:844] Serialized 52 byte log entry
0709 12:11:54.117075 (+   221us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:54.117337 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 12:11:54.285922 (+168585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.286031 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.267599 (+981568us) raft_consensus.cc:1685] finished
0709 12:11:55.267626 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.267814 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175221,"prepare.queue_time_us":601689,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":4683,"raft.queue_time_us":192316,"raft.run_cpu_time_us":612,"raft.run_wall_time_us":94171,"thread_start_us":624093,"threads_started":2,"wal-append.queue_time_us":963363}
I20260709 12:11:55.228621 15700 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:55.227305 15774 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.228603 15687 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.232110 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:11:55.243913 15795 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:11:55.308872 11215 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:11:55.249758 15459 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:11:55.311489 15799 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:11:55.312983 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2913) took 1623 ms. Trace:
I20260709 12:11:55.313258 15549 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.237079 15734 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.313376 10586 rpcz_store.cc:276] 0709 12:11:53.689150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.775177 (+1086027us) service_pool.cc:224] Handling call
0709 12:11:54.775603 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.803134 (+ 27531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.803193 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.803981 (+   788us) log.cc:844] Serialized 26 byte log entry
0709 12:11:55.096430 (+292449us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:55.096481 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.096566 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.312727 (+216161us) raft_consensus.cc:1685] finished
0709 12:11:55.312751 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.312897 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70643,"raft.queue_time_us":58911,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":40546,"thread_start_us":203151,"threads_started":1,"wal-append.queue_time_us":345439}
I20260709 12:11:55.314869 15734 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.317595 10586 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:55.319552 10586 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:11:55.300176 15774 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:11:55.326892 15812 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:11:55.328377 10585 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:55.354900 15808 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 12:11:55.375912 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.376758 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.381479 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2642) took 1257 ms. Trace:
I20260709 12:11:55.382004 10579 rpcz_store.cc:276] 0709 12:11:54.123883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.832924 (+709041us) service_pool.cc:224] Handling call
0709 12:11:54.833342 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.904100 (+ 70758us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:55.006404 (+102304us) log.cc:844] Serialized 14 byte log entry
0709 12:11:55.228153 (+221749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:55.381108 (+152955us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:55.381161 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.381253 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.381399 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":94723,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":5611,"raft.queue_time_us":11740,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"spinlock_wait_cycles":67429632,"thread_start_us":118698,"threads_started":1,"wal-append.queue_time_us":178702}
W20260709 12:11:55.383630 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.383574 15786 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:55.389866 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 170717us
I20260709 12:11:55.396200 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:55.396404 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 187667us
W20260709 12:11:55.402221 10553 leader_election.cc:336] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.709s (SENT)
I20260709 12:11:55.404982 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.405905 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.406338 11220 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:11:55.412606 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 362047us
I20260709 12:11:55.392885 10583 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.413713 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2528) took 1807 ms. Trace:
I20260709 12:11:55.408212 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:55.415150 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:55.389546 15808 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.417984 11215 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:55.419150 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.396713 15705 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.420747 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:55.422827 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:55.423053 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:55.418943 10553 leader_election.cc:336] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.738s (SENT)
I20260709 12:11:55.431946 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.393289 15813 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:11:55.442207 15813 raft_consensus.cc:2764] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:11:55.443895 15685 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.411700 15801 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:55.418376 15810 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:55.445878 10553 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.793s (SENT)
I20260709 12:11:55.432612 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:55.409582 15549 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:55.451247 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.416685 15762 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.451541 11219 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:55.452086 14901 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:55.436012 15742 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:55.446687 15705 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.453006 15742 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:55.453625 10553 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.891s (SENT)
I20260709 12:11:55.454453 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.418540 15712 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:55.455142 15549 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.455420 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:55.455825 15549 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.440572 15698 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:55.456418 15741 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:55.457084 15741 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.457540 15741 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:55.458211 15549 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:55.447302 10511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2657) took 1763 ms (client timeout 1678 ms). Trace:
W20260709 12:11:55.459328 10511 rpcz_store.cc:269] 0709 12:11:53.684392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.827722 (+143330us) service_pool.cc:224] Handling call
0709 12:11:55.446978 (+1619256us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b540036e554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:55.447192 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811268864}
I20260709 12:11:55.435266 15790 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:11:55.455576 15712 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:55.468096 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:55.469110 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.418563 15701 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:55.431152 15787 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:55.456269 14901 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:55.458173 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.927s (SENT)
I20260709 12:11:55.448442 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2909) took 1760 ms. Trace:
I20260709 12:11:55.478037 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.478065 10584 rpcz_store.cc:276] 0709 12:11:53.687915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.722199 (+1034284us) service_pool.cc:224] Handling call
0709 12:11:54.785794 (+ 63595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.825869 (+ 40075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.448222 (+622353us) spinlock_profiling.cc:243] Waited 63.1 ms on lock 0x7b5400090950. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:55.448367 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145232128}
I20260709 12:11:55.430927 15734 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.454406 15742 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.476334 15712 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:55.491541 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.474773 15705 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:55.499245 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 101630us
I20260709 12:11:55.395529 15786 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:55.486609 15698 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:55.459131 15807 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.133s
I20260709 12:11:55.429454 15808 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.492738 15783 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.528440 15695 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:55.529093 15695 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:55.452122 15762 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:55.538357 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.544018 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 121114us
I20260709 12:11:55.546593 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:55.548462 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:55.549293 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.561862 15787 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:11:55.566696 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.567687 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.568964 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.570425 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.572400 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:55.556082 15709 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.844s	user 0.011s	sys 0.010s
I20260709 12:11:55.574370 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:55.584699 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.588454 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.585996 15783 consensus_queue.cc:1059] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:11:55.590677 15818 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:55.577131 15701 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:55.585879 15549 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:11:55.594900 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.613796 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:55.596278 15802 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
I20260709 12:11:55.597381 15790 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:11:55.599624 15811 log.cc:927] Time spent T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.351s	user 0.000s	sys 0.002s
W20260709 12:11:55.588361 15407 consensus_meta.cc:287] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.827s	user 0.014s	sys 0.007s
W20260709 12:11:55.586961 15584 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.826s	user 0.017s	sys 0.006s
I20260709 12:11:55.573894 15801 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:55.618305 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:55.568356 15808 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:55.623173 15824 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:11:55.623329 15804 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:11:55.621996 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.623903 15740 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.629199 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2816) took 1134 ms. Trace:
W20260709 12:11:55.629690 15805 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.330s	user 0.004s	sys 0.000s
W20260709 12:11:55.629601 10508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 568) took 2353 ms (client timeout 1786 ms). Trace:
I20260709 12:11:55.570384 15789 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:55.644483 10508 rpcz_store.cc:269] 0709 12:11:53.276464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.276613 (+   149us) service_pool.cc:224] Handling call
0709 12:11:55.629359 (+2352746us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400152554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:55.629491 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2936243456}
I20260709 12:11:55.626573 15787 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:55.647899 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:55.554597 15695 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:55.637038 15584 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.681931 15787 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:55.631518 15734 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.683871 15584 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:55.684779 15762 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.631162 15705 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:55.685613 15705 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:55.633623 15698 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:55.686476 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:55.686721 15698 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.687180 15698 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:55.688117 15695 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:55.641489 15407 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:55.694402 15407 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:55.695456 15407 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.699467 15687 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:55.700062 15687 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:55.700861 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.704917 15822 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.705507 15822 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.708012 15822 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.655186 15790 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:55.627104 15783 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:55.660161 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2527) took 2210 ms. Trace:
I20260709 12:11:55.661108 15690 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.662911 15700 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.667877 15711 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.669261 15572 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.670403 15717 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.672641 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.675120 15701 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.684873 10585 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:55.746840 10435 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
W20260709 12:11:55.656747 15816 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:11:55.761642 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 139737us
W20260709 12:11:55.763149 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.728817 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:55.730103 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:55.736526 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92824.2us
I20260709 12:11:55.736925 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:55.682873 15817 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:11:55.742563 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:55.741817 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:55.732211 10509 rpcz_store.cc:276] 0709 12:11:53.449374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.824853 (+1375479us) service_pool.cc:224] Handling call
0709 12:11:54.825320 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.913956 (+ 88636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.913997 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.914553 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:11:55.090167 (+175614us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:55.090220 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.327438 (+237218us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b540039f8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:55.327489 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.659784 (+332295us) raft_consensus.cc:1685] finished
0709 12:11:55.659811 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.660079 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":996,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":11933,"spinlock_wait_cycles":202839424,"thread_start_us":175192,"threads_started":1,"wal-append.queue_time_us":432478}
W20260709 12:11:55.759548 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 87039.3us
I20260709 12:11:55.768790 10434 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:55.642913 10430 rpcz_store.cc:276] 0709 12:11:54.494978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.495856 (+   878us) service_pool.cc:224] Handling call
0709 12:11:54.526681 (+ 30825us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.780176 (+253495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.780242 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.824363 (+ 44121us) log.cc:844] Serialized 26 byte log entry
0709 12:11:55.136272 (+311909us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:55.136310 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.136392 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.595435 (+459043us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:55.628832 (+ 33397us) raft_consensus.cc:1685] finished
0709 12:11:55.628859 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.629105 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64367,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":49761,"spinlock_wait_cycles":18335488,"thread_start_us":473098,"threads_started":2,"wal-append.queue_time_us":317377}
I20260709 12:11:55.764685 10429 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:55.763844 10431 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:55.765575 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:55.765928 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:55.734014 15584 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:55.683478 15734 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.691272 15740 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.772600 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:55.748834 15790 consensus_queue.cc:1243] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:11:55.770751 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.811527 10433 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:55.731191 15783 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:55.775463 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:55.776402 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.784510 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:55.813409 10581 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:11:55.766408 15701 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.785595 10435 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:55.731191 15762 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.761127 15807 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.787961 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:55.819855 11212 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:11:55.788880 15705 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:11:55.789772 15695 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:55.777434 15729 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:55.827096 15740 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.827363 15729 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:55.806777 15690 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.828011 15729 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:55.796164 15766 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.951s	user 0.009s	sys 0.014s
I20260709 12:11:55.831729 15766 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.833040 15818 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:55.777456 15789 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:11:55.836459 10509 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:55.836544 11220 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
I20260709 12:11:55.837604 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:55.797798 15739 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:55.731766 15788 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.848076 15788 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:55.766606 15727 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:55.802224 15801 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:55.803493 15700 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.805250 15734 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:55.807536 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:55.808068 15711 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.850369 15519 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:55.808998 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:55.414131 10507 rpcz_store.cc:276] 0709 12:11:53.605913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.874507 (+1268594us) service_pool.cc:224] Handling call
0709 12:11:54.909340 (+ 34833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.909633 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:11:55.130288 (+220655us) log.cc:844] Serialized 14 byte log entry
0709 12:11:55.413320 (+283032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:55.413379 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:55.413447 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.413529 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.413646 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":294375,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":90,"thread_start_us":434057,"threads_started":2}
I20260709 12:11:55.853835 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:55.802229 15459 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.859870 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:55.837604 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:55.823664 10431 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:55.885178 10438 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:55.888384 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.810012 15822 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:55.891438 10436 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:55.892120 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.886300 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:55.892824 10509 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:11:55.902822 10505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 567) took 2627 ms (client timeout 1931 ms). Trace:
I20260709 12:11:55.850754 15701 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:55.914605 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2640) took 2293 ms. Trace:
I20260709 12:11:55.914775 10431 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:11:55.911363 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.915714 10429 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:55.915479 10509 rpcz_store.cc:276] 0709 12:11:53.620775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.892047 (+2271272us) service_pool.cc:224] Handling call
0709 12:11:55.892486 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:55.914238 (+ 21752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.914531 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:55.913496 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:11:55.902084 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.760257 15717 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.928184 10438 raft_consensus.cc:2393] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:55.930516 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:55.945227 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:55.784999 15549 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 12:11:55.932253 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:55.932579 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:55.950234 10581 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:11:55.933430 11215 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:11:55.834839 15766 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:55.850828 15519 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:55.959491 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:55.900938 15790 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:11:55.962203 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.962805 15790 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:55.963320 15790 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:55.963532 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.964641 15790 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.939183 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:55.965193 15519 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:55.884176 15807 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:55.966621 15519 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:55.968618 15807 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:55.883961 15774 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.010s
I20260709 12:11:55.969477 15774 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:55.945699 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.939468 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.946061 10437 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:55.971832 10431 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:11:55.967016 11212 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:55.939813 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.965384 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:55.913321 15805 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:11:55.973299 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.977629 10511 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:55.973409 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 200885us
I20260709 12:11:55.913178 15801 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:55.914557 10505 rpcz_store.cc:269] 0709 12:11:53.275937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.276100 (+   163us) service_pool.cc:224] Handling call
0709 12:11:55.902525 (+2626425us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540038efd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:55.902705 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3092528512}
I20260709 12:11:55.853032 15762 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:55.973845 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:55.979594 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 579) took 2380 ms. Trace:
I20260709 12:11:55.981644 10553 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:11:56.018673 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:55.989959 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:56.033061 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51634us
I20260709 12:11:56.022667 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2637) took 2138 ms. Trace:
I20260709 12:11:56.000666 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.033689 10429 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:56.021876 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:56.027349 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 105971us
I20260709 12:11:56.033591 10431 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:11:56.033979 10587 rpcz_store.cc:276] 0709 12:11:53.884101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.822845 (+938744us) service_pool.cc:224] Handling call
0709 12:11:54.823292 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.899746 (+ 76454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.899800 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.900711 (+   911us) log.cc:844] Serialized 26 byte log entry
0709 12:11:55.130663 (+229952us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:55.161850 (+ 31187us) log.cc:844] Serialized 14 byte log entry
0709 12:11:55.162134 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.162246 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.475509 (+313263us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:56.014593 (+539084us) raft_consensus.cc:1685] finished
0709 12:11:56.022365 (+  7772us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.022574 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42134,"prepare.queue_time_us":33870,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":140567,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":25146,"spinlock_wait_cycles":11421440,"thread_start_us":132431,"threads_started":1,"wal-append.queue_time_us":236247}
I20260709 12:11:56.037753 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:56.038498 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.039779 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:56.040583 10505 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:56.040948 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:56.034224 10504 rpcz_store.cc:276] 0709 12:11:53.599276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.865752 (+1266476us) service_pool.cc:224] Handling call
0709 12:11:54.913998 (+ 48246us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:54.968346 (+ 54348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.968417 (+    71us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:54.998263 (+ 29846us) log.cc:844] Serialized 52 byte log entry
0709 12:11:55.411356 (+413093us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:55.411398 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.411481 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.979294 (+567813us) raft_consensus.cc:1685] finished
0709 12:11:55.979331 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.979508 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5666,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":6931,"thread_start_us":315143,"threads_started":1,"wal-append.queue_time_us":428707}
I20260709 12:11:56.042385 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:56.045426 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2532) took 2370 ms. Trace:
I20260709 12:11:56.045848 10513 rpcz_store.cc:276] 0709 12:11:53.675337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.044607 (+2369270us) service_pool.cc:224] Handling call
0709 12:11:56.044925 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:56.045112 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:56.045135 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:56.045173 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:56.045203 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.045267 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.045365 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:56.064227 10513 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:11:56.066679 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:55.994889 15689 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.017s	sys 0.016s
I20260709 12:11:56.069764 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 581) took 2386 ms. Trace:
I20260709 12:11:56.072619 10513 rpcz_store.cc:276] 0709 12:11:53.682881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.056109 (+2373228us) service_pool.cc:224] Handling call
0709 12:11:56.056544 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:56.069047 (+ 12503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.069655 (+   608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:56.010740 15761 raft_consensus.cc:2764] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:11:56.094864 15789 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:56.095937 15683 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.122s	user 0.014s	sys 0.011s
I20260709 12:11:55.953702 15808 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 12:11:56.074388 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 136537us
I20260709 12:11:55.901886 15787 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:56.089375 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.082465 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.069247 15709 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:56.074024 10505 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:11:56.154516 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2647) took 2531 ms. Trace:
I20260709 12:11:56.154891 10505 rpcz_store.cc:276] 0709 12:11:53.622873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.043818 (+2420945us) service_pool.cc:224] Handling call
0709 12:11:56.073815 (+ 29997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:56.154181 (+ 80366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.154436 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:56.108718 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:56.156894 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:56.157097 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.157541 10431 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:56.140939 10513 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:11:56.072185 15689 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.017s	sys 0.023s
I20260709 12:11:56.072333 15790 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.146944 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:56.147105 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64755.4us
I20260709 12:11:56.150315 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:56.180028 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:56.173352 10431 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:56.174587 10432 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:56.175621 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 582) took 1583 ms. Trace:
I20260709 12:11:56.177376 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.182212 10429 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:11:56.183326 10513 rpcz_store.cc:276] 0709 12:11:54.591697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.076044 (+1484347us) service_pool.cc:224] Handling call
0709 12:11:56.140506 (+ 64462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:56.159612 (+ 19106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.175477 (+ 15865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:56.188612 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.190421 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.192312 10513 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:11:56.193408 10505 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:56.040459 15801 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:56.156594 15840 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:11:56.048060 15698 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.194311 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2538) took 1455 ms. Trace:
I20260709 12:11:56.220094 10505 rpcz_store.cc:276] 0709 12:11:54.738884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.155758 (+1416874us) service_pool.cc:224] Handling call
0709 12:11:56.193141 (+ 37383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:56.194001 (+   860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.194242 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:56.193399 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2540) took 1444 ms. Trace:
I20260709 12:11:56.221325 10513 rpcz_store.cc:276] 0709 12:11:54.748571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.184591 (+1436020us) service_pool.cc:224] Handling call
0709 12:11:56.191855 (+  7264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:56.193037 (+  1182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.193319 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:56.196487 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.223002 10431 raft_consensus.cc:2393] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:56.104684 15833 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:55.959120 15783 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.206744 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.208143 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.215107 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:56.236714 10429 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:11:56.201959 15687 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
W20260709 12:11:56.184528 15836 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:11:56.159420 15789 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:56.240911 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:56.244788 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.248394 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2525) took 2819 ms. Trace:
I20260709 12:11:56.241387 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:56.235800 10436 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:11:56.252663 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.248816 10512 rpcz_store.cc:276] 0709 12:11:53.429340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.816857 (+1387517us) service_pool.cc:224] Handling call
0709 12:11:54.829891 (+ 13034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:55.249398 (+419507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.248113 (+998715us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b540038f9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:56.248322 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934661888}
I20260709 12:11:56.245572 10432 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:56.258824 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.205950 15822 raft_consensus.cc:2764] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:11:56.206234 15818 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:56.215725 15690 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:11:56.289256 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 109339us
I20260709 12:11:56.290138 10505 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:56.187119 15683 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:11:56.299430 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:56.301826 11178 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.679s (SENT)
I20260709 12:11:56.265161 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:56.298494 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.303565 11213 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:56.236708 15459 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.246793 15705 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.259385 15810 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:56.322281 10508 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:11:56.260754 15789 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.317049 10431 raft_consensus.cc:2410] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:11:56.330070 10432 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:56.330595 10432 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:56.314119 15741 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.856s	user 0.016s	sys 0.002s
I20260709 12:11:56.311833 10429 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:56.260011 15687 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.318399 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.308362 15584 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:56.288546 15807 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:56.352283 10505 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:56.311396 15717 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.339159 10432 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.325939 15822 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.324107 15833 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:56.369200 15807 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:56.370443 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:56.371214 10432 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:56.375958 15762 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:56.376137 15831 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
W20260709 12:11:56.377058 15832 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
W20260709 12:11:56.379777 15834 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:11:56.358134 15741 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:56.384994 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:56.336601 15789 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.341327 15549 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:56.392045 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 137784us
W20260709 12:11:56.393368 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 325946us
I20260709 12:11:56.399734 11212 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:11:56.403682 11215 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:56.362767 15572 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:56.352991 15584 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:56.333382 15705 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:56.395529 15712 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.891s	user 0.001s	sys 0.013s
W20260709 12:11:56.463420 15695 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.002s
I20260709 12:11:56.408921 15822 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:56.464073 15695 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:56.463594 15712 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:56.465862 15822 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:56.466177 15712 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:56.469470 15814 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.198s	user 0.000s	sys 0.006s
W20260709 12:11:56.429152 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 187865us
W20260709 12:11:56.470876 15842 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:11:56.405501 15833 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.471612 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.472260 15833 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.472937 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2529) took 2857 ms. Trace:
I20260709 12:11:56.473114 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.474272 15762 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:56.473352 10506 rpcz_store.cc:276] 0709 12:11:53.615464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.103419 (+1487955us) service_pool.cc:224] Handling call
0709 12:11:55.133426 (+ 30007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:55.133985 (+   559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:55.134032 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:55.141900 (+  7868us) log.cc:844] Serialized 26 byte log entry
0709 12:11:55.227132 (+ 85232us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:55.227177 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.227719 (+   542us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:56.468782 (+1241063us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:56.472609 (+  3827us) raft_consensus.cc:1685] finished
0709 12:11:56.472633 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.472867 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11536,"spinlock_wait_cycles":13325056,"thread_start_us":80793,"threads_started":1,"wal-append.queue_time_us":93090}
I20260709 12:11:56.475005 15762 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:56.475181 15741 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:56.413261 15645 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.476117 15645 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:56.477415 15801 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:56.471663 10511 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 2542) took 1310 ms (client timeout 1525 ms). Trace:
I20260709 12:11:56.479426 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:56.480484 10511 rpcz_store.cc:269] 0709 12:11:55.161143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.467740 (+306597us) service_pool.cc:224] Handling call
0709 12:11:56.471443 (+1003703us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54000da554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:56.471594 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156802944}
I20260709 12:11:56.479707 10510 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:56.484072 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:56.455993 15584 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:56.488149 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:56.488677 11218 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:11:56.489234 15774 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.489918 11216 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEARNER]: Advancing to term 14
I20260709 12:11:56.439613 15549 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:56.470795 15705 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:56.490409 15519 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.001s
W20260709 12:11:56.513469 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.481660 15572 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:56.507114 10432 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
W20260709 12:11:56.517601 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:56.520293 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.518234 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:56.523888 10432 pending_rounds.cc:85] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 9
I20260709 12:11:56.524354 10432 pending_rounds.cc:107] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:11:56.524739 10432 raft_consensus.cc:2889] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:56.529505 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.505514 15549 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.504616 15701 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:11:56.577467 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.508863 15762 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:56.595816 15752 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:56.596334 15752 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:56.594295 15818 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:56.596781 15752 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.597026 15818 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.599093 15695 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:56.531579 15800 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:56.600140 15800 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.602406 15800 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:56.527431 15645 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:56.504153 15810 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:11:56.610217 11216 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:56.614796 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.616086 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.594620 15701 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.623281 10587 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:56.624630 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:56.616811 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.626655 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:56.617298 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:56.624198 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89186us
I20260709 12:11:56.618036 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.618317 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.592195 15856 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:56.526207 15786 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 12:11:56.608767 15858 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.617538 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:11:56.637784 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.625999 10585 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:56.641661 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.645690 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.651198 10586 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:56.680392 10580 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:56.685233 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.686336 10587 raft_consensus.cc:2410] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:11:56.629848 15701 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.626421 15810 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:56.640447 15856 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.702625 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.705042 10587 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:11:56.706585 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:56.690411 15739 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.005s
I20260709 12:11:56.697551 15818 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.620498 15690 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:11:56.703621 10585 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
W20260709 12:11:56.716220 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 190535us
I20260709 12:11:56.706274 15856 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:56.730806 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:56.741309 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:56.708742 15687 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.685797 15858 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.722579 15807 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.752874 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:56.755242 10585 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:56.735962 15705 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:56.757506 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78177.6us
I20260709 12:11:56.759722 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.761327 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.776726 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:56.780104 10511 raft_consensus.cc:2368] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:56.729203 15810 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:11:56.784426 10512 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:56.784044 14901 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:56.791707 10478 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:56.795455 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:56.704828 15572 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.796772 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:56.799000 10505 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:56.802955 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.799602 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:56.812597 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:56.823215 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:56.824406 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:56.764673 15852 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.763397 15705 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.809056 15687 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:56.851466 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67421.6us
I20260709 12:11:56.825930 15847 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:11:56.828338 15727 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:56.833164 15778 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:56.853695 15853 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 12:11:56.869606 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:56.871392 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.856449 15872 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:56.873417 15792 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:11:56.873652 15872 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:56.878381 15801 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:56.879010 15801 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:11:56.879755 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:56.880326 15870 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 12:11:56.884315 15866 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 12:11:56.885378 15868 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:11:56.885972 15729 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.057s	user 0.014s	sys 0.000s
W20260709 12:11:56.887100 15860 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:11:56.894759 15698 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:56.896678 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.854326 15687 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.901872 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:56.896574 15807 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:56.903863 10429 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:56.830132 15690 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:11:56.911016 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:56.912667 10436 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:11:56.886340 15865 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:11:56.915560 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.886420 15729 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.866447 15727 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.945362 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.959232 15727 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:56.952492 10552 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.525s (SENT) [suppressed 38 similar messages]
I20260709 12:11:56.877048 15872 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:56.879771 15858 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:56.954705 10580 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:56.960992 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.961704 10430 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:56.956280 15729 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:56.962869 15729 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:56.868672 15778 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:56.963804 15778 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.965464 15729 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:56.966315 15729 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:56.880015 15801 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:56.970837 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:56.971638 10429 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:56.913067 15783 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:56.914716 15739 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:57.003870 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:57.015228 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.029932 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:57.031989 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:56.913723 15687 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:56.948922 15788 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:57.038089 10512 raft_consensus.cc:2410] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:11:57.040956 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.979238 15709 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.053324 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:57.055383 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.058470 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:57.060138 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:57.061511 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:57.062237 10506 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:11:57.065696 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:56.947990 15459 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
W20260709 12:11:57.078805 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 275907us
I20260709 12:11:56.914430 15833 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.074174 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:57.080217 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:57.047993 15788 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:57.139385 15788 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:57.139923 15711 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:57.139274 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65213.7us
I20260709 12:11:57.145771 15869 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:57.146270 15869 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:11:57.147300 15869 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.148850 15869 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.124655 15698 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:57.152899 15833 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:57.154354 15789 consensus_meta.cc:287] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.001s
I20260709 12:11:57.154893 15789 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.155980 15774 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:57.157171 15789 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:57.158268 15856 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.158311 10579 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:57.160488 15856 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:57.066180 15709 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.162685 15709 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:57.052891 15784 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:11:57.164624 10587 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:11:57.063482 15712 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:11:57.099858 10508 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2659) took 1721 ms (client timeout 1631 ms). Trace:
W20260709 12:11:57.167969 10508 rpcz_store.cc:269] 0709 12:11:55.378326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.686152 (+307826us) service_pool.cc:224] Handling call
0709 12:11:57.099551 (+1413399us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b540039f3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:57.099745 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052931840}
I20260709 12:11:57.039147 15807 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:11:57.169261 15807 consensus_queue.cc:1243] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:11:57.071326 15847 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:57.106635 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2644) took 3484 ms. Trace:
I20260709 12:11:57.173115 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 2274) took 1025 ms. Trace:
I20260709 12:11:57.171986 10509 rpcz_store.cc:276] 0709 12:11:53.622028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.917868 (+2295840us) service_pool.cc:224] Handling call
0709 12:11:55.918312 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:56.138640 (+220328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:56.140694 (+  2054us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:56.141979 (+  1285us) log.cc:844] Serialized 26 byte log entry
0709 12:11:56.416953 (+274974us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:56.458454 (+ 41501us) log.cc:844] Serialized 14 byte log entry
0709 12:11:56.479047 (+ 20593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.479136 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:57.106346 (+627210us) raft_consensus.cc:1685] finished
0709 12:11:57.106373 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:57.106558 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30959,"prepare.queue_time_us":46062,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":130833,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":36733,"spinlock_wait_cycles":13525760,"thread_start_us":389607,"threads_started":2,"wal-append.queue_time_us":315667}
I20260709 12:11:57.173552 10431 rpcz_store.cc:276] 0709 12:11:56.148071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.234366 (+ 86295us) service_pool.cc:224] Handling call
0709 12:11:57.172908 (+938542us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b540034b054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:57.173048 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166099840}
W20260709 12:11:57.175911 10510 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 586) took 1756 ms (client timeout 1973 ms). Trace:
W20260709 12:11:57.176363 10510 rpcz_store.cc:269] 0709 12:11:55.419263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.838773 (+419510us) service_pool.cc:224] Handling call
0709 12:11:57.175677 (+1336904us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54000f91d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:57.175835 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1432183680}
I20260709 12:11:57.177655 10507 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:57.043949 15739 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.042255 15849 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:11:57.178836 15762 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:11:57.179306 15849 consensus_queue.cc:1243] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:11:57.179430 15762 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:57.179989 15849 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:11:57.180428 15849 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:57.180771 15762 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.181334 15849 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.182255 15762 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:57.185341 15739 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:57.117012 15848 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.702s	user 0.002s	sys 0.000s
I20260709 12:11:57.187371 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 586) took 1302 ms. Trace:
I20260709 12:11:57.187732 10437 rpcz_store.cc:276] 0709 12:11:55.885341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.978403 (+ 93062us) service_pool.cc:224] Handling call
0709 12:11:56.006559 (+ 28156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:56.006956 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:56.007295 (+   339us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:56.012813 (+  5518us) log.cc:844] Serialized 26 byte log entry
0709 12:11:56.185676 (+172863us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:56.416316 (+230640us) log.cc:844] Serialized 14 byte log entry
0709 12:11:56.416618 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.416719 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:57.116311 (+699592us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:57.187059 (+ 70748us) raft_consensus.cc:1685] finished
0709 12:11:57.187167 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:57.187302 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":467,"prepare.queue_time_us":280679,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":598,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":22022,"spinlock_wait_cycles":36070144,"thread_start_us":341661,"threads_started":2,"wal-append.queue_time_us":359131}
W20260709 12:11:57.189730 15867 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.410s	user 0.003s	sys 0.000s
W20260709 12:11:57.108342 15790 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.909s	user 0.009s	sys 0.006s
I20260709 12:11:57.190599 15790 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:57.123462 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.193252 10583 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:11:57.194377 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2659) took 1265 ms. Trace:
I20260709 12:11:57.194723 10583 rpcz_store.cc:276] 0709 12:11:55.929475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.929878 (+   403us) service_pool.cc:224] Handling call
0709 12:11:57.194196 (+1264318us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400334d54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:57.194299 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2822119168}
I20260709 12:11:57.195922 15712 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.196594 15700 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.196502 15712 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.197126 15700 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.199617 15700 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:57.201332 10511 raft_consensus.cc:2376] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12. Re-sending same reply.
I20260709 12:11:57.105893 15683 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:57.216850 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:57.217670 10430 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.219280 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:57.220434 10512 raft_consensus.cc:2410] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:11:57.221753 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:57.225131 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:57.233023 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:57.233816 10509 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:57.250095 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:57.250347 11216 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
W20260709 12:11:57.072886 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 83977.1us
I20260709 12:11:57.131273 10505 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:57.078161 15699 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:57.273528 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:57.267762 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 307669us
I20260709 12:11:57.182865 15849 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:11:57.252715 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:57.284066 10510 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:57.216500 15712 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:57.281940 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.781s (SENT)
I20260709 12:11:57.300949 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.303956 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:57.306221 10478 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:57.295571 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.122138 15701 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.287355 10504 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:57.290817 10580 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:57.243783 15786 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:57.329246 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:57.330166 10582 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:57.332226 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:57.226615 15790 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:57.340426 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.049s	sys 0.096s
I20260709 12:11:57.343953 10582 raft_consensus.cc:2410] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 12:11:57.346762 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.049s	sys 0.096s
I20260709 12:11:57.303213 10583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:11:57.353524 10583 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.354169 10583 raft_consensus.cc:614] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:57.356321 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:57.355480 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:57.128436 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:57.412079 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 283730us
I20260709 12:11:57.413909 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:57.415982 11178 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.400867 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:57.313612 15784 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:57.313889 15800 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:57.404285 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:57.405656 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:57.405972 10429 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:57.302141 10586 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:11:57.322610 15712 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.093s
W20260709 12:11:57.377615 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71455.5us
I20260709 12:11:57.286639 15699 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.442873 10432 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:57.436209 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
W20260709 12:11:57.351100 15584 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.001s
I20260709 12:11:57.352772 15690 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:57.347783 15790 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:57.466836 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62684.7us
I20260709 12:11:57.467907 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:57.404277 15858 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:57.407446 15783 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:57.349299 15786 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.427016 15709 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:57.456040 15800 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:57.492753 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:57.468950 15856 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:57.456969 15818 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:57.471541 15761 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:57.505396 15709 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.507233 15810 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:57.511420 15712 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:57.537040 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 395194us
I20260709 12:11:57.471163 15584 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.511580 15807 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:57.511762 15808 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:57.509951 15784 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:11:57.519037 15778 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.555s	user 0.013s	sys 0.001s
I20260709 12:11:57.507000 15883 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:57.516721 15711 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:57.477833 15858 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.504092 15783 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.517261 15884 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:57.574875 10430 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:57.535384 15709 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:11:57.552459 15857 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.010s	sys 0.039s
W20260709 12:11:57.605770 15857 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.010s	sys 0.039s
W20260709 12:11:57.567548 15882 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:11:57.606639 10434 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:57.579139 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.536854 15810 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:57.546892 15807 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:57.625716 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.593047 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.476775 15690 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.571246 15784 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.589048 15741 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:57.566941 15705 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:57.567988 15881 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:11:57.603508 15884 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.545261 15790 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:57.549988 15818 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.520772 15856 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:57.648578 10435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.007s	sys 0.020s
W20260709 12:11:57.651306 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60830.8us
W20260709 12:11:57.611654 15885 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
W20260709 12:11:57.686257 10435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.007s	sys 0.020s
W20260709 12:11:57.650825 15752 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.054s	user 0.014s	sys 0.003s
I20260709 12:11:57.635363 15807 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:57.691639 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.550865 15712 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:11:57.633553 15810 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:57.650570 15838 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:11:57.590013 15778 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:11:57.889994 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 198483us
I20260709 12:11:57.724502 15884 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:57.680969 15741 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:57.758697 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:57.897106 11219 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:57.633962 15800 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.744609 15701 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:57.755316 15783 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:57.796000 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.764434 15889 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:57.855638 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:57.903057 11177 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.010s	sys 0.005s
I20260709 12:11:57.714615 15711 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:57.889096 10552 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.001s	sys 0.013s
I20260709 12:11:57.701715 15787 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.891013 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:57.839190 15858 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:57.893316 15894 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:11:57.905360 10552 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.001s	sys 0.013s
W20260709 12:11:57.911841 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 959396us
W20260709 12:11:57.915802 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.882812 15752 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:57.903911 11177 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.010s	sys 0.006s
W20260709 12:11:57.918219 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 704882us
W20260709 12:11:57.905033 15459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.012s	sys 0.034s
W20260709 12:11:57.888795 15801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.007s	sys 0.009s
I20260709 12:11:57.937636 10586 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:57.931528 10582 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:57.949792 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:57.923774 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:57.952210 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:57.953711 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.903128 15786 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:57.908247 15852 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.054s	user 0.011s	sys 0.011s
I20260709 12:11:58.034705 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:57.921792 15584 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:57.937110 15572 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:58.054047 15572 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.031867 15847 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:58.055451 15847 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.055827 15847 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:58.056759 15572 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:57.986418 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.057642 10582 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:58.059208 15683 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:11:57.919620 15800 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:58.060449 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.061437 10578 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:58.054261 15584 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:57.981220 11219 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:11:58.075163 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.150513 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:58.063542 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2926) took 1448 ms. Trace:
I20260709 12:11:58.151589 10579 rpcz_store.cc:276] 0709 12:11:56.615307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.615441 (+   134us) service_pool.cc:224] Handling call
0709 12:11:58.063253 (+1447812us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b5400336654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:58.063452 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220521344}
I20260709 12:11:58.163537 15572 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:58.164439 15572 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.167321 15572 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:58.146508 15786 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.172490 15786 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:57.937289 15808 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:11:57.917961 15784 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:58.182907 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.036s	sys 0.053s
W20260709 12:11:58.194609 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.037s	sys 0.053s
W20260709 12:11:57.945883 15801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.008s	sys 0.011s
W20260709 12:11:58.189342 10438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.018s	sys 0.073s
W20260709 12:11:57.977206 15459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.015s	sys 0.034s
I20260709 12:11:57.957485 15690 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:58.195041 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 678712us
I20260709 12:11:58.292842 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:57.910209 15893 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:11:58.196189 11216 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.025s	sys 0.054s
I20260709 12:11:58.295493 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:57.909966 15889 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:58.199628 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.153290 15727 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.193s	user 0.016s	sys 0.006s
W20260709 12:11:58.201339 10438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.018s	sys 0.073s
W20260709 12:11:58.202481 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.206023 11178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.027s	sys 0.092s
W20260709 12:11:58.309834 11178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.027s	sys 0.092s
W20260709 12:11:58.310518 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 894573us
I20260709 12:11:58.311579 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:11:58.208735 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.191450 15892 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.292s	user 0.001s	sys 0.001s
I20260709 12:11:57.910682 15787 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.175433 15701 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:58.312848 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.318837 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:58.318585 10436 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:58.320930 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.321188 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.321767 10432 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:11:57.931214 15739 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.264s
I20260709 12:11:57.949342 15705 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.242s
I20260709 12:11:58.193615 15572 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:58.301401 15761 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:11:58.300223 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2530) took 4520 ms. Trace:
I20260709 12:11:58.301939 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.367501 10507 rpcz_store.cc:276] 0709 12:11:53.661132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.058711 (+2397579us) service_pool.cc:224] Handling call
0709 12:11:57.177439 (+1118728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.178080 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.181319 (+1003239us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54000f91d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:58.181519 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571981568}
I20260709 12:11:58.209519 15894 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:58.204599 15818 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:58.373746 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 323776us
W20260709 12:11:58.314932 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 366255us
I20260709 12:11:58.322857 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.377161 10432 raft_consensus.cc:2410] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:11:58.323591 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:58.323904 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.330302 10436 raft_consensus.cc:2410] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:11:58.332530 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 345852us
I20260709 12:11:58.291536 15786 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:58.335047 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.295231 11216 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.026s	sys 0.054s
I20260709 12:11:58.314205 15905 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:58.354709 10437 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:11:58.315877 15690 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:58.201166 15852 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:58.357744 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.383024 15897 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.488s	user 0.000s	sys 0.001s
W20260709 12:11:58.367491 15891 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.482s	user 0.001s	sys 0.000s
I20260709 12:11:58.383395 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.439188 10510 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:58.383631 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.384622 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.385653 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.440676 10436 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:58.441665 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.441836 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.442703 10432 raft_consensus.cc:2393] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:11:58.356586 15572 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.346474 15683 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:58.318006 15727 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.419629 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2859) took 1438 ms. Trace:
I20260709 12:11:58.419375 10433 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:58.454708 10581 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:58.455324 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2861) took 1431 ms. Trace:
I20260709 12:11:58.388726 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.202780 15808 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.430197 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.436127 10508 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:58.369837 15407 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:58.390661 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.378676 15705 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:58.443526 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:58.424716 15549 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.813s	user 0.012s	sys 0.006s
I20260709 12:11:58.386731 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.384758 15894 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:58.445717 15909 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:11:58.446380 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.418675 15883 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:58.443931 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 411107us
I20260709 12:11:58.447587 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:58.475999 10582 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:58.450186 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.477087 10436 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.453320 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.420080 15905 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.454734 10437 rpcz_store.cc:276] 0709 12:11:56.980857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.189823 (+208966us) service_pool.cc:224] Handling call
0709 12:11:58.353930 (+1164107us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:11:58.354197 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.419276 (+ 65079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.419551 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11490688}
I20260709 12:11:58.391407 15818 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.460702 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.455811 10433 rpcz_store.cc:276] 0709 12:11:57.024044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.195843 (+171799us) service_pool.cc:224] Handling call
0709 12:11:58.418589 (+1222746us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:11:58.418868 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.454941 (+ 36073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.455247 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26065792}
W20260709 12:11:58.443176 15908 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.160s	user 0.001s	sys 0.001s
I20260709 12:11:58.473284 10579 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:58.513139 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:58.515976 15739 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:58.516645 15898 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 12:11:58.554306 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 172033us
I20260709 12:11:58.517495 15913 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:58.442786 15900 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:58.524732 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.571651 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.572032 10508 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:58.454228 15712 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:58.477346 15901 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.540s	user 0.000s	sys 0.001s
I20260709 12:11:58.389621 15778 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:11:58.467419 15690 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:58.523729 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:11:58.548622 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 229841us
I20260709 12:11:58.466486 15705 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:58.466356 15752 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:58.550561 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:58.555737 11177 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.786s (SENT)
I20260709 12:11:58.556558 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 812) took 1347 ms. Trace:
W20260709 12:11:58.526670 15869 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.377s	user 0.014s	sys 0.003s
I20260709 12:11:58.559162 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:58.580004 10432 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:58.581151 10505 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:58.559952 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.515964 15917 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:58.562304 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.569941 10436 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:11:58.554189 15786 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:58.561319 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:58.527639 15905 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.572312 10433 raft_consensus.cc:2376] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13. Re-sending same reply.
I20260709 12:11:58.477519 15407 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.621256 15407 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:58.623375 15572 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:58.623375 10435 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:11:58.575158 15900 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.627270 15900 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:58.627231 15407 raft_consensus.cc:476] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 12:11:58.584131 15712 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:58.630509 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 917) took 1291 ms. Trace:
I20260709 12:11:58.579627 15918 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:58.631639 15918 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.631253 10580 rpcz_store.cc:276] 0709 12:11:57.338793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.338957 (+   164us) service_pool.cc:224] Handling call
0709 12:11:57.339358 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:57.339895 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.339948 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:57.357105 (+ 17157us) log.cc:844] Serialized 52 byte log entry
0709 12:11:57.716701 (+359596us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:57.716748 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.716827 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.630202 (+913375us) raft_consensus.cc:1685] finished
0709 12:11:58.630235 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.630425 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":11884,"thread_start_us":256250,"threads_started":1,"wal-append.queue_time_us":471539}
I20260709 12:11:58.634688 15923 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.575047 15801 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:11:58.635454 15923 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.480141 15549 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.588327 15690 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.638453 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.639081 15786 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.639720 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.640566 10586 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:58.640297 15915 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:11:58.641390 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.629735 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:58.642204 10586 raft_consensus.cc:2393] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:58.642313 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.643077 10436 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:58.578552 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 185415us
W20260709 12:11:58.585409 15902 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:11:58.586544 15910 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:11:58.507889 15852 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:11:58.591557 10437 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:11:58.582592 11177 leader_election.cc:336] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.931s (SENT)
I20260709 12:11:58.582755 11215 rpcz_store.cc:276] 0709 12:11:57.209177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.209336 (+   159us) service_pool.cc:224] Handling call
0709 12:11:57.209787 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.210202 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.210237 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:57.210756 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:11:57.536715 (+325959us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:57.536771 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.536863 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.556255 (+1019392us) raft_consensus.cc:1685] finished
0709 12:11:58.556280 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.556465 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1225,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":17633,"thread_start_us":325490,"threads_started":1,"wal-append.queue_time_us":545510}
I20260709 12:11:58.594731 11212 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:58.572754 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.598745 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:58.600512 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.603782 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.562491 15913 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:58.576336 15784 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:58.614367 10508 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:58.605485 15741 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.616017 15727 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:58.595918 15788 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.456s	user 0.009s	sys 0.006s
I20260709 12:11:58.701407 15788 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.702100 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.702270 10579 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:58.634280 15705 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.703578 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.703893 15705 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:58.705032 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.705672 10579 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:58.585080 15889 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:58.616070 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:58.645735 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.647477 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:58.707702 10586 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:58.708830 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:58.709590 10586 raft_consensus.cc:2393] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:11:58.710668 10586 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:58.553946 15818 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.711198 10585 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:11:58.711319 15761 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:58.711968 15761 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.712757 15818 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.713088 10586 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:11:58.714197 15761 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:58.714453 15788 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:58.691273 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:11:58.715689 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 133132us
W20260709 12:11:58.716372 11177 leader_election.cc:336] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.933s (SENT)
I20260709 12:11:58.716563 15808 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:58.716807 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:58.717186 15808 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:58.665750 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104527us
I20260709 12:11:58.717849 15808 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:58.718180 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:58.691838 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.601127 15783 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:58.564435 15739 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.692996 10505 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:58.697980 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.701735 15884 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:58.702735 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:58.707559 11219 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:58.712833 10437 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:58.664757 10432 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:11:58.706647 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:11:58.731854 10579 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:58.707346 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 596) took 1357 ms. Trace:
I20260709 12:11:58.733476 10512 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:58.641480 15915 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:58.724035 10508 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:58.671726 10436 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:58.714335 15818 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.727205 10580 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:58.697552 15800 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.001s
I20260709 12:11:58.628973 15917 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:58.737035 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.692124 15549 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.732577 10438 rpcz_store.cc:276] 0709 12:11:57.349995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.350153 (+   158us) service_pool.cc:224] Handling call
0709 12:11:57.353744 (+  3591us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.406407 (+ 52663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.406460 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:57.407390 (+   930us) log.cc:844] Serialized 26 byte log entry
0709 12:11:58.309257 (+901867us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:58.309325 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.309440 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.707108 (+397668us) raft_consensus.cc:1685] finished
0709 12:11:58.707136 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.707281 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":975,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":19812,"thread_start_us":795372,"threads_started":1,"wal-append.queue_time_us":963552}
I20260709 12:11:58.696835 15784 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.714824 15761 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.742432 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:11:58.641346 15683 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.697253 15929 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:58.748134 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.743378 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:58.774449 10505 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:58.630568 15712 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.778710 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:11:58.726060 15727 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:11:58.728960 15884 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:58.765156 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.767057 15407 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.738286 15921 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.729586 15925 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:11:58.751649 15915 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:58.758698 15741 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:11:58.772291 15917 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:58.734217 15783 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:58.763473 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:58.776283 10582 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:11:58.780596 11217 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:58.782291 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:11:58.783036 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.769986 10403 leader_election.cc:400] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13.
I20260709 12:11:58.779095 15929 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.775822 15784 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:11:58.788493 15687 consensus_meta.cc:287] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.732s	user 0.009s	sys 0.008s
W20260709 12:11:58.801973 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.798647 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.742306 15858 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:58.800884 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.804375 15917 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.790169 15778 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:58.817008 15917 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.820437 15801 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:58.821208 15884 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:58.826398 15709 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.222s	user 0.012s	sys 0.009s
W20260709 12:11:58.831609 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.833500 10505 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:58.766260 15818 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:58.838670 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 68892.4us
I20260709 12:11:58.842829 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:58.797246 10579 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:58.850085 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.847584 11219 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:58.791706 15690 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:58.852403 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:58.801533 15929 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:58.870756 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.807557 15914 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:11:58.871798 10505 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:58.788341 15761 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:58.875548 15933 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:11:58.837782 15711 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:58.813853 15918 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.894809 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2832) took 3114 ms. Trace:
I20260709 12:11:58.861124 15709 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.879827 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2856) took 1933 ms. Trace:
I20260709 12:11:58.879915 15884 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:58.794332 15407 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.880566 15877 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:58.880857 15894 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.881094 15786 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.792393 15921 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.771831 15913 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.837486 15923 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:58.886561 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:58.889490 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:58.911548 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.889909 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.912510 11219 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:11:58.861676 15801 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:58.895745 15711 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.895298 10434 rpcz_store.cc:276] 0709 12:11:55.773308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.773450 (+   142us) service_pool.cc:224] Handling call
0709 12:11:55.798484 (+ 25034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.607131 (+1808647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.887575 (+1280444us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400436a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:58.887768 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4157912320}
I20260709 12:11:58.895532 10429 rpcz_store.cc:276] 0709 12:11:56.946296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.972463 (+ 26167us) service_pool.cc:224] Handling call
0709 12:11:56.972852 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.457181 (+484329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.879500 (+1422319us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540015e5d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:58.879751 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":925796736}
I20260709 12:11:58.863197 15778 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:58.814000 15687 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.905642 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:11:58.913354 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 140347us
W20260709 12:11:58.897801 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.932291 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.842168 15858 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.918516 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2552) took 1689 ms. Trace:
I20260709 12:11:58.936499 10512 rpcz_store.cc:276] 0709 12:11:57.229538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.230499 (+   961us) service_pool.cc:224] Handling call
0709 12:11:57.230856 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.733962 (+1503106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.918163 (+184201us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400413a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:58.918426 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3455248128}
I20260709 12:11:58.937819 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.938582 10512 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:11:58.933252 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:58.943728 10512 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:58.944547 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.944552 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:58.931710 15711 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:58.957163 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.958371 10436 raft_consensus.cc:2410] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:11:58.959369 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.959941 10436 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:11:58.960611 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.960842 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.922335 15786 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:11:58.905375 15849 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.605s	user 0.012s	sys 0.008s
W20260709 12:11:58.906304 15847 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.850s	user 0.015s	sys 0.000s
I20260709 12:11:58.925890 10511 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:58.932837 15877 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:11:58.927953 15923 consensus_queue.cc:1059] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:58.904768 15690 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:58.961999 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:11:58.964437 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.930752 15913 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.965083 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:58.969143 10513 raft_consensus.cc:2368] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14.
I20260709 12:11:59.003479 11219 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:11:58.921231 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 177907us
I20260709 12:11:58.928141 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:58.971697 10437 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:58.992564 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:58.992354 15847 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.993643 10508 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:11:58.926637 15549 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:11:58.998135 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:58.994071 15889 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:59.005534 10438 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:11:58.968832 15701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.006s	sys 0.002s
I20260709 12:11:59.006721 15858 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.039435 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.044739 10505 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:59.002384 15852 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:58.970820 15925 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:11:59.045296 15709 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.046550 15407 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.073118 15761 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.074165 15801 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:59.065935 15701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.007s	sys 0.003s
I20260709 12:11:59.079700 15701 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:59.046821 15847 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.047495 15572 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.048429 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:59.035261 15778 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:59.050525 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 122455us
I20260709 12:11:58.969792 15876 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:11:58.972651 15739 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:58.920382 15783 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:58.915686 15935 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:11:59.063153 10436 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:59.064014 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57706.9us
I20260709 12:11:59.064183 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.068313 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.068660 15858 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:59.069131 15712 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:59.072402 15699 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:59.072319 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:59.123147 10438 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:11:59.082885 10432 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:11:59.083475 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:11:59.089207 15807 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 1.204s	user 0.012s	sys 0.002s
I20260709 12:11:59.121749 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.080372 15852 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.089543 15572 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.123457 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:59.139130 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.139918 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:59.085068 15876 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.141686 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.132694 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.165674 10579 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:11:59.167029 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:11:59.143569 15883 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.637s	user 0.013s	sys 0.000s
W20260709 12:11:59.144714 15896 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 1.425s	user 0.000s	sys 0.005s
I20260709 12:11:59.173107 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:59.148140 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2957) took 1217 ms. Trace:
I20260709 12:11:59.175169 10581 rpcz_store.cc:276] 0709 12:11:57.931085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.933055 (+  1970us) service_pool.cc:224] Handling call
0709 12:11:59.147906 (+1214851us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b5400208fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:59.148055 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":963471104}
I20260709 12:11:59.152319 10509 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:11:59.155624 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 921) took 1699 ms. Trace:
I20260709 12:11:59.176813 10436 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:11:59.176465 10584 rpcz_store.cc:276] 0709 12:11:57.456263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.456426 (+   163us) service_pool.cc:224] Handling call
0709 12:11:57.475147 (+ 18721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.605657 (+130510us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:11:57.605996 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 12:11:58.313722 (+707726us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:58.313789 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:58.313843 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.155369 (+841526us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400276ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:59.155420 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:59.155557 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18901,"prepare.queue_time_us":26188,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":147864,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":115,"spinlock_wait_cycles":299531136,"thread_start_us":688086,"threads_started":2,"wal-append.queue_time_us":727460}
I20260709 12:11:59.158540 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.069492 15889 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:11:59.183537 15941 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:11:59.187255 15883 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:59.191071 15905 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:59.348970 15905 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:59.351050 15407 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:59.161358 15729 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.160066 15699 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:59.160854 15871 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:11:59.196247 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.358860 10579 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:11:59.196587 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:59.363065 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 166612us
I20260709 12:11:59.081785 15929 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.207114 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.157871 15923 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:59.208725 10432 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:59.209424 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 45762.2us
W20260709 12:11:59.212524 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.171741 15876 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:59.231937 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.372088 15923 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.233043 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.233929 15700 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:11:59.235592 15931 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.412s	user 0.003s	sys 0.000s
I20260709 12:11:59.243139 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:11:59.400635 10430 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:11:59.400763 11213 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 813) took 2166 ms. Trace:
I20260709 12:11:59.252013 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:11:59.276340 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:11:59.283133 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2551) took 2059 ms. Trace:
I20260709 12:11:59.402370 10506 rpcz_store.cc:276] 0709 12:11:57.223268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.223416 (+   148us) service_pool.cc:224] Handling call
0709 12:11:57.223723 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.503333 (+279610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.503372 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:57.503418 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:57.503449 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.282769 (+1779320us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540039eed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:59.282835 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:59.283045 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642499328}
I20260709 12:11:59.404496 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.405930 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.288666 15687 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.293578 15945 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.295056 15915 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.298074 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.298470 15930 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:59.298122 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.421037 10510 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
W20260709 12:11:59.302187 15904 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.920s	user 0.002s	sys 0.000s
I20260709 12:11:59.238708 15856 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:59.304934 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:59.112309 15877 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.307435 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:59.308331 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.428910 10507 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:59.387194 15549 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:59.322863 15787 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.214511 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:59.324193 10513 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 2546) took 2129 ms (client timeout 1937 ms). Trace:
I20260709 12:11:59.351679 15407 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.327499 15927 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:59.328763 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2959) took 1394 ms. Trace:
I20260709 12:11:59.343622 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.163249 15913 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:59.346056 15572 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:59.346720 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:11:59.448839 10584 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:59.353458 11221 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:11:59.352056 10433 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 13. Ops: []
I20260709 12:11:59.192116 15701 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.368153 10580 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:11:59.270749 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.159372 15858 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:11:59.375422 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.453020 15927 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:59.218266 10479 connection.cc:256] RPC call timeout handler was delayed by -1.6428557760000331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
I20260709 12:11:59.454120 15927 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.455276 15927 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:59.459446 15584 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:59.460089 15584 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.462126 15801 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:59.463410 15858 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.467024 15584 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:59.467909 15883 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:59.468349 15883 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:11:59.383136 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.384737 15889 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:59.401247 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48986 (request call id 2282) took 1098 ms. Trace:
I20260709 12:11:59.470490 10430 rpcz_store.cc:276] 0709 12:11:58.303821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.304059 (+   238us) service_pool.cc:224] Handling call
0709 12:11:59.242870 (+938811us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:11:59.401176 (+158306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11916672}
I20260709 12:11:59.428663 15930 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:59.210273 15847 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:59.410964 10553 connection.cc:256] RPC call timeout handler was delayed by -1.4932511150000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:11:59.486720 10553 connection.cc:256] RPC call timeout handler was delayed by -1.4194511149999016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
W20260709 12:11:59.487200 10553 connection.cc:256] RPC call timeout handler was delayed by -1.3123511149999558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17370000000000002s for a response
W20260709 12:11:59.487488 10553 connection.cc:256] RPC call timeout handler was delayed by -1.2952511149999282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
I20260709 12:11:59.420962 15711 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.489470 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:11:59.488155 10553 connection.cc:256] RPC call timeout handler was delayed by -1.1665511149999475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:11:59.490814 10553 connection.cc:256] RPC call timeout handler was delayed by -1.1323511149998922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 12:11:59.491137 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0783511150000322s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:11:59.491400 10553 connection.cc:256] RPC call timeout handler was delayed by -1.0495511149999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 12:11:59.491686 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:11:59.411613 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 135377us
I20260709 12:11:59.200233 15818 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.413918 11215 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:11:59.511212 15847 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:59.512615 15847 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:11:59.516968 15407 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.428303 15945 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:59.485371 15810 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.575s	user 0.010s	sys 0.008s
I20260709 12:11:59.532405 15810 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:59.414300 10431 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 2858) took 2425 ms (client timeout 2480 ms). Trace:
W20260709 12:11:59.536585 10431 rpcz_store.cc:269] 0709 12:11:56.976731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.174586 (+197855us) service_pool.cc:224] Handling call
0709 12:11:58.306089 (+1131503us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:11:59.251793 (+945704us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bae9c 00007f2c8f9baa5b 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97aa50a0 00007f2c97aa4984 00007f2c9101f6d4 00007f2c9101f662
0709 12:11:59.401799 (+150006us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27260416}
W20260709 12:11:59.387429 15784 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.001s
I20260709 12:11:59.420163 10436 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:59.401230 11213 rpcz_store.cc:276] 0709 12:11:57.209508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.209634 (+   126us) service_pool.cc:224] Handling call
0709 12:11:57.209971 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.210330 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.210360 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:57.210836 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:11:57.511077 (+300241us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:57.511145 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.511251 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:59.144019 (+1632768us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:11:59.355245 (+211226us) raft_consensus.cc:1685] finished
0709 12:11:59.355282 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:59.376242 (+ 20960us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64859,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":17951,"spinlock_wait_cycles":52622080,"thread_start_us":299805,"threads_started":1,"wal-append.queue_time_us":488161}
I20260709 12:11:59.358352 15699 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.423360 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.468777 15883 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:59.389818 15923 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.425916 10509 raft_consensus.cc:2376] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16. Re-sending same reply.
I20260709 12:11:59.547627 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.306023 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.548892 10504 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:11:59.370664 15871 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:59.399717 15838 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:59.430066 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.309689 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.534837 15810 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.400112 15700 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:59.557551 15871 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.559185 15871 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:59.560794 15852 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:59.430927 10505 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:11:59.423039 15917 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
W20260709 12:11:59.435796 10513 rpcz_store.cc:269] 0709 12:11:57.194584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.214250 (+ 19666us) service_pool.cc:224] Handling call
0709 12:11:59.323954 (+2109704us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540036f454. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:11:59.324105 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4003144064}
I20260709 12:11:59.554179 10507 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:11:59.326833 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:11:59.442550 10585 rpcz_store.cc:276] 0709 12:11:57.933873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.935521 (+  1648us) service_pool.cc:224] Handling call
0709 12:11:57.935902 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.711666 (+775764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.328450 (+616784us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400213254. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:59.328679 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782551296}
I20260709 12:11:59.386440 15729 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:59.447425 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 233047us
I20260709 12:11:59.476882 15929 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.406646 10506 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
W20260709 12:11:59.485054 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.142118 15807 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.492152 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:59.545094 15699 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:59.544458 15687 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:59.424314 15690 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:11:59.500419 15943 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:59.501219 15741 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:59.501631 15889 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:59.525714 15945 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:11:59.540107 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 49101us
I20260709 12:11:59.542632 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.591688 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:59.543506 15949 raft_consensus.cc:1064] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:11:59.542863 15930 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.441644 15549 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.556128 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:11:59.644762 10401 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.015s	sys 0.022s
I20260709 12:11:59.645171 10509 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:11:59.557462 15838 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:59.558744 15700 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:59.433835 15900 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.806s	user 0.016s	sys 0.000s
I20260709 12:11:59.654141 15700 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:59.653849 15900 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.657086 15915 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:59.657375 15900 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.490141 15911 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:59.642082 15800 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:11:59.659439 15911 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.659582 15800 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.662967 11214 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:11:59.541626 15921 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.664294 15711 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.665069 15911 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:11:59.574877 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 82813.2us
I20260709 12:11:59.670955 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:11:59.579262 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.672986 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:59.672226 15584 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 4 to 10, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:11:59.674356 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:59.674423 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:59.675194 10513 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:59.675908 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:59.170312 15786 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.237s
I20260709 12:11:59.593780 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.685150 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.685817 10511 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:11:59.594046 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.686803 10504 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:11:59.614472 11216 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:59.544826 15784 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:11:59.645243 10401 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.015s	sys 0.022s
I20260709 12:11:59.694118 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2286) took 1376 ms. Trace:
I20260709 12:11:59.694684 10433 rpcz_store.cc:276] 0709 12:11:58.317299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.654115 (+336816us) service_pool.cc:224] Handling call
0709 12:11:58.695500 (+ 41385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:59.455342 (+759842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.693781 (+238439us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400166cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:59.694035 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509367552}
W20260709 12:11:59.696108 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:59.644204 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52654.1us
I20260709 12:11:59.697865 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 709) took 1094 ms. Trace:
I20260709 12:11:59.580616 15852 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:59.768131 10508 pending_rounds.cc:85] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 11
I20260709 12:11:59.649574 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.615054 15889 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.648893 15945 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:59.648519 15741 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:11:59.494912 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.680254 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:59.686528 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:11:59.690467 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:59.760224 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 187106us
I20260709 12:11:59.751412 15788 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:11:59.641430 15949 raft_consensus.cc:1068] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:11:59.698376 11214 rpcz_store.cc:276] 0709 12:11:58.603317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.603462 (+   145us) service_pool.cc:224] Handling call
0709 12:11:58.603823 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:59.663468 (+1059645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.697545 (+ 34077us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400821c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:59.697785 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435239680}
W20260709 12:11:59.775449 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 78554.4us
I20260709 12:11:59.798094 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:59.755448 15884 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.852s	user 0.012s	sys 0.003s
I20260709 12:11:59.677832 15930 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.729034 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2958) took 1795 ms. Trace:
I20260709 12:11:59.582600 15687 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.788777 10508 pending_rounds.cc:107] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:11:59.729012 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:11:59.791692 10510 raft_consensus.cc:2410] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:11:59.751992 15915 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:59.731717 15783 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:59.803350 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 123217us
W20260709 12:11:59.803704 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 113379us
I20260709 12:11:59.798581 15800 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:11:59.749598 15955 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:11:59.850080 15955 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.838485 15774 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:59.799213 15929 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.769299 15911 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:59.826377 10583 rpcz_store.cc:276] 0709 12:11:57.933521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.933653 (+   132us) service_pool.cc:224] Handling call
0709 12:11:57.937223 (+  3570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.937492 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:58.076564 (+139072us) log.cc:844] Serialized 14 byte log entry
0709 12:11:59.277271 (+1200707us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b1800358420. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f23a 00007f2c92ae9a2a
0709 12:11:59.727648 (+450377us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:59.727721 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:59.727782 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.728785 (+  1003us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:59.728965 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138801,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"spinlock_wait_cycles":175028608,"thread_start_us":440145,"threads_started":1}
I20260709 12:11:59.828490 10508 raft_consensus.cc:2889] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:59.836548 10511 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:11:59.837489 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.838548 10510 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:59.838910 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 266810us
I20260709 12:11:59.843701 11216 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 706) took 1249 ms. Trace:
I20260709 12:11:59.837855 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:11:59.835093 15687 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.843199 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:59.818428 15903 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:59.835090 15858 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:59.867280 15712 consensus_queue.cc:1059] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:11:59.868703 15876 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.869163 15807 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:59.870747 15889 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:11:59.794703 15945 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:11:59.842123 15783 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:11:59.912572 15783 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:59.917140 15801 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.917729 15801 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:11:59.899137 15952 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
W20260709 12:11:59.894054 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50964.9us
I20260709 12:11:59.924651 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:59.889027 11216 rpcz_store.cc:276] 0709 12:11:58.594175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.595209 (+  1034us) service_pool.cc:224] Handling call
0709 12:11:58.603905 (+  8696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:59.687802 (+1083897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.843400 (+155598us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400622854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:11:59.843630 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323461248}
W20260709 12:11:59.896389 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58641.9us
I20260709 12:11:59.927105 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:11:59.870647 15923 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.858358 15741 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.907266 15954 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.897770 15778 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:59.897961 15858 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.911360 15729 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:11:59.889621 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.901145 15949 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:59.929121 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:11:59.928635 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:11:59.947328 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.817992 15838 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.783931 15690 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.852175 15929 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:11:59.911445 15549 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:11:59.982366 10513 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:11:59.922739 15687 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:11:59.950954 15963 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:59.950840 15956 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:00.135967 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 206948us
I20260709 12:12:00.138087 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:00.029838 15949 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.140625 15921 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:00.141639 15699 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 12:11:59.984323 15858 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:00.144801 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:00.034243 15914 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:12:00.169314 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:00.032518 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 103952us
W20260709 12:12:00.206084 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.227772 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:11:59.982000 15934 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:12:00.221405 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76691.5us
W20260709 12:12:00.204787 15701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.029s
I20260709 12:12:00.234511 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:00.225028 15900 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:00.237875 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:00.204711 15549 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:00.224908 15762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.011s	sys 0.011s
I20260709 12:12:00.216176 15889 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.268724 15889 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:00.271921 15801 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:00.274555 15801 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:00.252183 15900 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:00.279397 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2665) took 2990 ms. Trace:
I20260709 12:12:00.279645 15900 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:00.279901 10587 rpcz_store.cc:276] 0709 12:11:57.288987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.289125 (+   138us) service_pool.cc:224] Handling call
0709 12:11:57.289491 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.306286 (+ 16795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.306324 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.128510 (+1822186us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b180048ee00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a
0709 12:11:59.272510 (+144000us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:11:59.273256 (+   746us) log.cc:844] Serialized 26 byte log entry
0709 12:11:59.571463 (+298207us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:59.615375 (+ 43912us) log.cc:844] Serialized 14 byte log entry
0709 12:11:59.615702 (+   327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.615819 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:59.850098 (+234279us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b180048ee00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:00.275877 (+425779us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b180048ee00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:00.279099 (+  3222us) raft_consensus.cc:1685] finished
0709 12:12:00.279126 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.279287 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":69479,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":27958,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":11818,"spinlock_wait_cycles":47568384,"thread_start_us":297691,"threads_started":1,"wal-append.queue_time_us":478675}
I20260709 12:12:00.282500 15917 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:00.251766 15913 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:00.232511 15699 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:12:00.287133 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.235491 15810 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:00.208143 15949 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:00.031312 15945 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:00.259109 15969 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:12:00.226897 15786 raft_consensus.cc:2764] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:12:00.225651 15914 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.260277 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2869) took 1301 ms. Trace:
I20260709 12:12:00.266319 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2566) took 1586 ms. Trace:
W20260709 12:12:00.266734 15961 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:12:00.241771 15822 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:12:00.218461 15778 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:00.237093 15701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.008s	sys 0.029s
I20260709 12:12:00.236109 15711 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:00.233187 15876 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:00.234547 15784 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:00.233001 15934 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:00.299679 10438 rpcz_store.cc:276] 0709 12:11:58.958372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.123822 (+165450us) service_pool.cc:224] Handling call
0709 12:11:59.160084 (+ 36262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:59.162545 (+  2461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.162583 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.172378 (+  9795us) log.cc:844] Serialized 26 byte log entry
0709 12:11:59.667878 (+495500us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b1800382c00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1 00007f2c92a9aa0f
0709 12:11:59.956274 (+288396us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:59.956335 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.956446 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.259892 (+303446us) raft_consensus.cc:1685] finished
0709 12:12:00.259923 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.260198 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36886,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":178568,"spinlock_wait_cycles":35004160,"thread_start_us":288159,"threads_started":1,"wal-append.queue_time_us":1046113}
I20260709 12:12:00.290148 15699 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:00.315325 10505 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.022s	sys 0.087s
W20260709 12:12:00.316565 10505 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.022s	sys 0.087s
W20260709 12:12:00.334431 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:00.297758 10550 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.023s
W20260709 12:12:00.336979 10553 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.004s	sys 0.015s
W20260709 12:12:00.336467 10550 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.006s	sys 0.023s
I20260709 12:12:00.299878 10512 rpcz_store.cc:276] 0709 12:11:58.681204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.959656 (+278452us) service_pool.cc:224] Handling call
0709 12:11:58.960190 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:58.970291 (+ 10101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:58.970347 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.351019 (+380672us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4c00044240. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c90daba20 00007f2c90dbca29 00007f2c90d82644 00007f2c90da9f2d 00007f2c90da9b93 00007f2c92ada18b 00007f2c92adacb5 00007f2c92adc004 00007f2c92adb729 00007f2c92ae0d96 00007f2c92ad5adb
0709 12:11:59.375930 (+ 24911us) log.cc:844] Serialized 487 byte log entry
0709 12:11:59.832596 (+456666us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:59.832645 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.833559 (+   914us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.265889 (+432330us) raft_consensus.cc:1685] finished
0709 12:12:00.265922 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.266225 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73844096,"thread_start_us":398829,"threads_started":1,"wal-append.queue_time_us":774468}
I20260709 12:12:00.293270 15956 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:00.349928 10553 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.004s	sys 0.015s
I20260709 12:12:00.305394 15945 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:00.309537 15711 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.302888 15913 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:00.357913 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 682056us
W20260709 12:12:00.351022 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.302317 15943 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:59.975598 15954 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:00.301708 15914 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:00.398458 15883 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.829s	user 0.014s	sys 0.000s
W20260709 12:12:00.424655 10578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2675) took 1978 ms (client timeout 1772 ms). Trace:
W20260709 12:12:00.430377 10578 rpcz_store.cc:269] 0709 12:11:58.446047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.603492 (+157445us) service_pool.cc:224] Handling call
0709 12:12:00.411737 (+1808245us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b540010aed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:00.424497 (+ 12760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092395904}
I20260709 12:12:00.310213 15778 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:00.357213 15956 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:00.434401 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.449627 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:00.368659 15889 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:00.370362 15711 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:00.454154 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2862) took 1498 ms. Trace:
I20260709 12:12:00.461299 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:00.374840 15923 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:00.468863 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:00.470103 10552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.057s	sys 0.087s
I20260709 12:12:00.412979 15955 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:00.472970 10552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.057s	sys 0.087s
I20260709 12:12:00.473847 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:00.476320 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:00.478725 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:00.463177 10434 rpcz_store.cc:276] 0709 12:11:58.955932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:58.960143 (+  4211us) service_pool.cc:224] Handling call
0709 12:11:59.259767 (+299624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:59.260101 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:59.575671 (+315570us) log.cc:844] Serialized 14 byte log entry
0709 12:11:59.981099 (+405428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:59.981204 (+   105us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:59.981286 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.981397 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.453881 (+472484us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b540012ff50. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:00.454067 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105831,"prepare.queue_time_us":203895,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":24374,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":37252,"spinlock_wait_cycles":661239552,"thread_start_us":698334,"threads_started":2,"wal-append.queue_time_us":691403}
W20260709 12:12:00.480773 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 236426us
I20260709 12:12:00.535081 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:00.535935 10431 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:00.455683 15956 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:00.541556 10437 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:00.482774 10587 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:12:00.537025 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:00.486948 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:00.433557 15913 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:00.554540 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:00.482157 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2570) took 1031 ms. Trace:
I20260709 12:12:00.548933 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:00.555871 10581 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:12:00.556388 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.549453 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.490338 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:00.555069 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 76470.1us
I20260709 12:12:00.557067 10579 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:00.558090 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:00.556726 10504 rpcz_store.cc:276] 0709 12:11:59.450939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.687522 (+236583us) service_pool.cc:224] Handling call
0709 12:11:59.755579 (+ 68057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:59.756036 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.756080 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.756700 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:12:00.212003 (+455303us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:00.212069 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:00.212167 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.481867 (+269700us) raft_consensus.cc:1685] finished
0709 12:12:00.481894 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.482079 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":14887,"thread_start_us":392340,"threads_started":1,"wal-append.queue_time_us":567046}
I20260709 12:12:00.555240 10433 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:00.547780 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:00.561784 11214 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:12:00.564371 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:00.565773 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:00.569440 10438 raft_consensus.cc:2368] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:12:00.479462 15889 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:00.470417 15976 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:00.477695 15915 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:12:00.499403 15700 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.845s	user 0.016s	sys 0.001s
I20260709 12:12:00.587484 10432 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:00.587643 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.562215 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:00.590098 11216 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:00.443747 15883 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:00.605679 15883 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:00.431972 15934 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:00.606855 15934 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:00.577735 15889 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:00.607510 15889 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:00.580525 15584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:55454 (request call id 75) took 5934 ms. Trace:
I20260709 12:12:00.609716 15762 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:00.610152 15762 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:00.485448 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.576084 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:00.479736 15847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.015s	sys 0.043s
I20260709 12:12:00.490238 15784 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:00.578922 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.615773 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.617059 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.617612 10581 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:00.580137 11221 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:12:00.580684 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.622620 10580 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:00.581946 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:00.492900 15929 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:00.563174 15712 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:12:00.480969 15698 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:00.632006 10582 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:00.563030 15871 consensus_meta.cc:287] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.003s	user 0.012s	sys 0.003s
I20260709 12:12:00.586225 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:00.566237 15801 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:00.593532 15729 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:00.625430 10434 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:00.611598 15980 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:12:00.619294 15915 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.636719 15915 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:00.589509 15687 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:00.601224 15700 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.639708 15700 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:00.641882 15700 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:00.493381 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:00.570928 10578 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:12:00.631417 15847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.016s	sys 0.043s
I20260709 12:12:00.642930 10581 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:00.670532 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.670698 10582 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:00.673976 10581 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:00.671788 15883 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:00.692667 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:00.693535 10587 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
W20260709 12:12:00.695724 15973 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.284s	user 0.000s	sys 0.004s
I20260709 12:12:00.694901 10587 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:00.680091 15915 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:00.698719 10587 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:01.002275 15838 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:00.881559 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:12:00.954747 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.679041 10580 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:00.922595 10438 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:00.873354 10336 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:00.953162 10552 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:00.891423 10433 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:12:00.938333 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:00.875388 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.015s	sys 0.008s
W20260709 12:12:00.879719 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:00.882179 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:00.685468 15778 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:00.669121 15858 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:00.691071 15883 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.054062 15883 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:00.697381 15911 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:01.056524 15911 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:01.057806 15701 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:00.769755 15913 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:00.776707 15927 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.320s	user 0.016s	sys 0.001s
W20260709 12:12:00.813055 15949 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
I20260709 12:12:01.061260 15949 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:01.061874 15921 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.003s
I20260709 12:12:01.062568 15921 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:00.820545 15956 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.063100 10436 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:00.655937 15877 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:00.822528 15976 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:01.064586 15877 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:01.064966 15976 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:00.646270 15784 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:00.791309 15982 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.384s	user 0.002s	sys 0.001s
W20260709 12:12:00.770329 15988 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:12:00.644075 10579 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:00.647768 15762 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.078433 15762 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:01.079100 15847 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.876435 10431 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:01.079995 15847 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:00.648751 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:00.912271 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:01.008927 15977 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:12:01.013188 10552 leader_election.cc:403] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:01.018750 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.021035 10433 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:00.608724 15584 rpcz_store.cc:276] 0709 12:11:54.642273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.264350 (+622077us) service_pool.cc:224] Handling call
0709 12:11:55.492293 (+227943us) log.cc:844] Serialized 487 byte log entry
0709 12:11:56.400656 (+908363us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18000d0480. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:00.563809 (+4163153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21509,"raft.queue_time_us":22091,"raft.run_cpu_time_us":928,"raft.run_wall_time_us":43348,"spinlock_wait_cycles":3930496,"thread_start_us":203060,"threads_started":1,"wal-append.queue_time_us":291997,"wal-append.run_cpu_time_us":2000,"wal-append.run_wall_time_us":1002005}
I20260709 12:12:01.023013 15741 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:01.023104 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.026180 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.019s	sys 0.032s
I20260709 12:12:01.035326 15917 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:01.002966 15838 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:00.648375 15965 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:12:01.036165 11177 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.014s	sys 0.014s
W20260709 12:12:00.647830 15975 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 12:12:01.045518 15912 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:00.666921 15801 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.066866 15784 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.071269 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:12:01.071630 10509 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 599) took 1616 ms (client timeout 1973 ms). Trace:
I20260709 12:12:01.073508 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 596) took 1619 ms. Trace:
I20260709 12:12:01.078547 10579 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:00.651650 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65512us
W20260709 12:12:01.086954 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.087949 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:01.087996 10429 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:12:01.099097 11177 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.015s	sys 0.015s
I20260709 12:12:01.066395 15877 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:01.097651 10509 rpcz_store.cc:269] 0709 12:11:59.454909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.649194 (+194285us) service_pool.cc:224] Handling call
0709 12:12:01.071056 (+1421862us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54000f91d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:01.071299 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2843664768}
I20260709 12:12:01.097981 10508 rpcz_store.cc:276] 0709 12:11:59.454016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.692258 (+238242us) service_pool.cc:224] Handling call
0709 12:11:59.767823 (+ 75565us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:59.887322 (+119499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.887375 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:59.898320 (+ 10945us) log.cc:844] Serialized 52 byte log entry
0709 12:12:00.300385 (+402065us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:00.300437 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:00.300526 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.751503 (+450977us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:01.072634 (+321131us) raft_consensus.cc:1685] finished
0709 12:12:01.072686 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:01.073167 (+   481us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9640,"raft.queue_time_us":141,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":6918,"spinlock_wait_cycles":46952320,"thread_start_us":388489,"threads_started":1,"wal-append.queue_time_us":501600}
I20260709 12:12:01.104225 15741 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.108322 15801 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.105540 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:01.100322 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 627143us
I20260709 12:12:01.111155 15741 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.115470 15912 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.105806 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:12:01.111187 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 876702us
I20260709 12:12:01.106335 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:01.119372 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:01.119586 11212 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:01.120914 15784 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:01.121717 15784 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:01.122231 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:01.122655 15784 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:01.135740 15985 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:01.058859 15701 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.093576 15883 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.142359 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:01.096827 15917 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.085708 15976 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:01.143579 11177 leader_election.cc:336] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.973s (SENT)
I20260709 12:12:01.172153 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:01.153280 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 473400us
I20260709 12:12:01.155489 15921 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:01.151466 15784 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:01.177363 10512 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:12:01.202179 15977 consensus_queue.cc:1243] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:12:01.165848 15858 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.193058 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:01.195554 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 597) took 1741 ms. Trace:
I20260709 12:12:01.196103 15883 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.188549 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:01.192530 15914 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.010s
I20260709 12:12:01.150451 15985 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:01.207844 15954 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:01.215919 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.216359 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.217162 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.221601 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 128519us
I20260709 12:12:01.223261 15847 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.224014 15976 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.225263 15687 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:01.201226 15921 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:01.226770 15687 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:01.227916 15687 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:01.230613 15985 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.223875 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:01.233557 15985 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.185462 15801 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.235110 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:01.236222 10508 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:01.238304 15956 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.239599 10586 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:01.243878 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:01.244944 11217 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:01.246945 15584 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:01.247618 15584 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:01.249445 15584 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.261639 15995 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:01.262351 10509 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:01.267315 15995 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:01.267887 15995 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:01.264066 15584 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:01.264645 15954 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:01.272186 15954 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.272881 15917 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.265421 15886 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:01.276216 15886 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.251746 15921 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.266582 15949 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.272639 15701 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.281023 15886 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.281296 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:01.282415 10512 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:01.281524 15954 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.229823 15687 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:01.295388 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 342702us
I20260709 12:12:01.282325 15784 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.304659 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:01.315133 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54985.2us
I20260709 12:12:01.224126 15847 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:12:01.334396 15847 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:01.334932 15847 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:01.207000 10505 rpcz_store.cc:276] 0709 12:11:59.454352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.693035 (+238683us) service_pool.cc:224] Handling call
0709 12:11:59.762408 (+ 69373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:59.762627 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:00.316977 (+554350us) log.cc:844] Serialized 14 byte log entry
0709 12:12:00.585482 (+268505us) log.cc:844] Serialized 14 byte log entry
0709 12:12:01.132312 (+546830us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:01.140662 (+  8350us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:01.143931 (+  3269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.152125 (+  8194us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:01.195288 (+ 43163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":918262,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":6186,"raft.queue_time_us":35628,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":181,"thread_start_us":1260826,"threads_started":3,"wal-append.queue_time_us":365764}
I20260709 12:12:01.336413 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:01.336925 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:01.337288 10506 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:01.344527 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:01.350162 15711 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.009s
I20260709 12:12:01.256068 15914 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.355459 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:01.356637 15914 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.358569 15883 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:01.365016 15549 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 1.113s	user 0.022s	sys 0.000s
I20260709 12:12:01.365751 15549 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:01.369894 11217 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:01.323627 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:01.324286 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.324951 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:01.385780 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71259.9us
I20260709 12:12:01.386862 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:01.387355 15838 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:01.326041 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 220964us
I20260709 12:12:01.327227 15912 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.331131 15690 raft_consensus.cc:1064] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:12:01.332804 15783 consensus_meta.cc:287] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.419s	user 0.019s	sys 0.001s
I20260709 12:12:01.418686 15783 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.419631 10584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:01.419806 15847 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.388314 15949 raft_consensus.cc:2764] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:12:01.388911 15985 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:01.396656 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.421679 15985 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:01.325251 15996 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.106s	user 0.000s	sys 0.002s
I20260709 12:12:01.422156 15985 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:01.424496 15847 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:01.427489 10579 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:01.350771 15711 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.431519 10504 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:01.432557 15711 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.397956 10479 leader_election.cc:400] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:12:01.436802 10479 leader_election.cc:403] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:12:01.437474 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58787.1us
I20260709 12:12:01.439400 15699 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:01.440431 15914 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:12:01.444649 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.447500 15921 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.451524 16001 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:01.452039 16001 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:01.452450 16001 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:01.455281 15584 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:01.455717 10336 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:01.456564 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 110099us
I20260709 12:12:01.457532 10336 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 15, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:01.460520 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:01.461467 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:01.461755 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:01.462502 11221 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 20: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:01.409085 10432 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:01.413343 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:01.464107 11213 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 4 to 10, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:01.465570 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52514.1us
I20260709 12:12:01.462608 11214 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:01.471349 15701 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:01.472090 15701 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:01.472515 15778 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.473274 15778 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.473443 15701 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.476323 15778 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.405997 11212 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
W20260709 12:12:01.481482 15889 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.873s	user 0.020s	sys 0.000s
I20260709 12:12:01.489020 10586 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:01.489678 10584 raft_consensus.cc:606] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:01.490141 10584 raft_consensus.cc:614] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:01.487362 10512 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:01.498880 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:01.409654 15886 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:01.500761 15995 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.421695 15783 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.450774 15921 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:01.478307 15911 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.513166 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:01.541966 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 44423.6us
I20260709 12:12:01.534593 15687 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:01.537719 15838 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:01.550033 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:01.498939 15889 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:01.564172 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 422441us
W20260709 12:12:01.570729 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.574355 10581 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.864s	user 0.013s	sys 0.002s
I20260709 12:12:01.574544 11221 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:01.562691 15762 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:12:01.580369 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.578436 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2986) took 1034 ms. Trace:
I20260709 12:12:01.583549 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:01.586853 15911 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.588080 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36746 (request call id 929) took 1103 ms. Trace:
I20260709 12:12:01.588498 10583 rpcz_store.cc:276] 0709 12:12:00.484590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.484734 (+   144us) service_pool.cc:224] Handling call
0709 12:12:00.490369 (+  5635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:00.490713 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:01.075298 (+584585us) log.cc:844] Serialized 14 byte log entry
0709 12:12:01.557627 (+482329us) log.cc:844] Serialized 14 byte log entry
0709 12:12:01.563953 (+  6326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:01.567383 (+  3430us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:01.577235 (+  9852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.587711 (+ 10476us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:01.587861 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2437,"prepare.queue_time_us":287796,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":2227,"raft.queue_time_us":17459,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":10293,"spinlock_wait_cycles":25472,"thread_start_us":853518,"threads_started":3,"wal-append.queue_time_us":351699}
I20260709 12:12:01.591495 15778 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:01.592252 15778 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:01.592686 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:01.593799 15886 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:01.593576 15778 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.596170 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:01.599280 10581 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:01.599395 15886 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:01.599968 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 2992) took 1054 ms. Trace:
I20260709 12:12:01.600332 10581 rpcz_store.cc:276] 0709 12:12:00.545643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.670173 (+124530us) service_pool.cc:224] Handling call
0709 12:12:01.599879 (+929706us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:01.601552 15915 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.602288 15915 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:01.603449 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:01.604661 15886 raft_consensus.cc:2764] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:12:01.605439 10585 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:12:01.607349 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 2975) took 1119 ms. Trace:
I20260709 12:12:01.607767 10585 rpcz_store.cc:276] 0709 12:12:00.488193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.488549 (+   356us) service_pool.cc:224] Handling call
0709 12:12:00.488967 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:01.606053 (+1117086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.606916 (+   863us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540010c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:01.607258 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2476096256}
I20260709 12:12:01.608590 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:01.609946 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.610953 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.581787 10586 rpcz_store.cc:276] 0709 12:12:00.543736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.572176 (+ 28440us) service_pool.cc:224] Handling call
0709 12:12:01.568097 (+995921us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400337054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:01.578337 (+ 10240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434336512}
I20260709 12:12:01.609623 11221 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:12:01.568994 10336 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 9 to 12, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:01.627748 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.623684 16002 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:12:01.638147 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.651376 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53290.6us
I20260709 12:12:01.656291 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:01.659232 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 182458us
I20260709 12:12:01.630379 15762 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:01.664875 10336 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:01.676222 15690 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:01.679646 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.683213 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 140693us
I20260709 12:12:01.686412 15889 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:01.687575 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:01.688779 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.689656 10429 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
W20260709 12:12:01.691152 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.694933 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.671833 15784 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 12:12:01.698048 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.696969 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:12:01.705158 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83765.9us
I20260709 12:12:01.733867 15762 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.739836 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:01.702718 15915 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:01.745862 10508 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.003s
I20260709 12:12:01.746308 10508 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:01.748096 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.748914 10433 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:01.763474 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:01.764252 11212 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:01.765858 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:12:01.770334 16015 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:12:01.771222 10333 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 224) took 28225 ms (client timeout 15000 ms). Trace:
I20260709 12:12:01.773010 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.760998 15699 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:12:01.784180 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.785175 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.796679 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.754374 15995 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.803615 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:01.759084 10336 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:01.805022 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:01.776284 15915 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:01.791916 15994 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:01.839210 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:01.781808 15913 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:12:01.844763 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 80493.9us
W20260709 12:12:01.847965 16014 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:12:01.848357 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.848487 15711 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:12:01.854389 15976 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.006s
I20260709 12:12:01.791373 15980 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.894424 15980 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.897217 15980 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:01.863108 16001 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:01.863947 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 114113us
I20260709 12:12:01.865938 15883 consensus_queue.cc:1059] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:01.902602 16001 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:01.903144 15883 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:12:01.903666 15883 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:01.904829 15883 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:01.906373 15883 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:01.867411 11217 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:12:01.771715 10333 rpcz_store.cc:269] 0709 12:11:33.506303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.506604 (+   301us) service_pool.cc:224] Handling call
0709 12:12:01.726811 (+28220207us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:53.072320 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:53.365254 (+292934us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:53.384224 (+ 18970us) write_op.cc:435] Acquired schema lock
0709 12:11:53.384245 (+    21us) tablet.cc:662] Decoding operations
0709 12:11:53.385252 (+  1007us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:53.385333 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:53.385363 (+    30us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:11:53.385954 (+   591us) tablet.cc:701] Row locks acquired
0709 12:11:53.385973 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:11:53.386140 (+   167us) write_op.cc:270] Start()
0709 12:11:53.386274 (+   134us) write_op.cc:276] Timestamp: P: 1783599113386083 usec, L: 0
0709 12:11:53.386297 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:11:53.468407 (+ 82110us) log.cc:844] Serialized 2446 byte log entry
0709 12:11:54.758734 (+1290327us) op_driver.cc:464] REPLICATION: finished
0709 12:11:55.300623 (+541889us) write_op.cc:301] APPLY: starting
0709 12:11:55.320275 (+ 19652us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:55.367915 (+ 47640us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:55.368060 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:55.462924 (+ 94864us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:55.485059 (+ 22135us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 12:11:55.485116 (+    57us) write_op.cc:312] APPLY: finished
0709 12:11:55.488428 (+  3312us) log.cc:844] Serialized 65 byte log entry
0709 12:11:55.570186 (+ 81758us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:55.573223 (+  3037us) write_op.cc:454] Released schema lock
0709 12:11:55.577752 (+  4529us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:55.636560 (+ 58808us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:55.637231 (+   671us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":204911,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":516659,"apply.run_cpu_time_us":54567,"apply.run_wall_time_us":345008,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":169709,"prepare.run_cpu_time_us":6015,"prepare.run_wall_time_us":460421,"replication_time_us":1372245,"thread_start_us":531542,"threads_started":3,"wal-append.queue_time_us":87055,"wal-append.run_cpu_time_us":1217,"wal-append.run_wall_time_us":1014414}]]}
I20260709 12:12:01.911237 15858 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:01.871418 15729 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.223s	user 0.015s	sys 0.007s
I20260709 12:12:01.914426 15729 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:01.929776 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.930583 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83637.4us
W20260709 12:12:01.933876 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 43645.3us
I20260709 12:12:01.926977 10336 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 9 to 14, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 14 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:01.937894 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:01.886596 15915 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:01.933403 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 94421.2us
I20260709 12:12:01.941236 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:01.943954 10583 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:12:01.944571 15729 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:01.945268 15729 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:01.945812 15994 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:01.947547 15994 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.950157 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:01.951332 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:01.952601 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:01.955876 15914 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:01.956322 15914 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:01.956708 15914 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:01.960517 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:01.961226 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.965602 11212 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
W20260709 12:12:01.959602 10478 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 2.182s (SENT) [suppressed 27 similar messages]
I20260709 12:12:01.968209 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:01.969237 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.979645 16001 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:01.980502 16001 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:01.942057 15915 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:01.976863 15985 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.554s	user 0.015s	sys 0.002s
I20260709 12:12:01.995268 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:01.995878 10512 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:01.996201 10586 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:01.997184 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:01.993829 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.004297 15858 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.946419 15729 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:01.990653 15687 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:02.011577 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.015890 11212 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:02.016294 10508 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:02.028944 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.029673 10581 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.029964 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.030354 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.031679 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.032460 10579 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 18: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:02.033300 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:01.991827 15741 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.020084 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:01.967351 15995 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:12:02.035682 15912 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:02.036592 15912 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:02.037009 15912 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:02.035418 15741 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.037698 11217 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.039412 15741 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.039685 10336 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 3 to 4, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 4 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:01.998329 15847 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:02.047062 10581 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:02.047308 15847 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:02.048369 15847 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.053328 11217 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
W20260709 12:12:02.056463 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.975589 15913 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.061311 15913 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.062458 10504 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:02.063897 11212 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:12:02.065729 15690 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:02.003646 15985 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.068457 15985 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:02.071259 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65328.6us
W20260709 12:12:02.062450 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.042119 10336 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 10 to 11, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:02.030803 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.981918 16001 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.079224 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 10 to 12, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:02.088584 10336 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 13 to 14, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 14 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:02.093176 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.027477 15687 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.099221 10583 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:02.097257 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.103511 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:02.104565 11217 raft_consensus.cc:2410] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:12:02.121348 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.123123 15858 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.123772 10508 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:02.126098 15858 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.129709 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:02.130512 11217 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:02.132109 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:02.132822 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:02.133249 11213 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:02.133462 15912 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:02.137651 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.152693 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:02.165937 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.167568 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.167768 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.168124 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:02.101105 15687 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:02.171434 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.171897 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:02.169099 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.172698 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:02.171447 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.174823 10433 raft_consensus.cc:2410] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:12:02.174705 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 102363us
I20260709 12:12:02.176429 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:02.170696 10336 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 6 to 8, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 8 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.155700 15858 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:02.184897 10430 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:02.187091 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.179292 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:02.185513 10434 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:02.185705 11221 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:02.100042 15729 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:02.186328 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.179893 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.189678 10435 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:02.190143 10431 raft_consensus.cc:2410] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:12:02.186532 10429 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:02.185930 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.191435 10585 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 12: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:02.191426 10583 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:12:02.195729 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.196410 10582 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:02.199293 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.200563 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.202263 10581 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:02.158720 15913 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.203603 15913 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.206372 15913 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.207791 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.217274 11221 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:12:02.217610 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e4cd5d330f694dacb199bd61266c8f5a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 12:12:02.219846 15912 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.221200 15912 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:02.223143 15858 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.225875 15858 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.234005 16000 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:02.234221 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.239454 10336 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:02.241482 10432 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:02.246806 10435 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
W20260709 12:12:02.247207 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.248279 10512 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:12:02.250830 10508 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:02.256883 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.257761 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:02.260222 10478 leader_election.cc:336] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Timed out: RequestConsensusVote RPC to 127.9.11.67:36727 timed out after 1.742s (SENT)
I20260709 12:12:02.266453 10581 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:02.271709 10506 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:12:02.272024 10433 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:02.273008 10579 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:02.274178 10510 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:02.275249 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.276611 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.248322 15883 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:12:02.278232 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 101904us
I20260709 12:12:02.279942 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:02.270763 10403 connection.cc:256] RPC call timeout handler was delayed by -1.8630960200000573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 12:12:02.284435 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 89801.1us
I20260709 12:12:02.288506 10581 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:02.292176 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.293313 10508 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:02.244143 15729 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.276458 15921 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:02.279239 15883 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:02.309356 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.313771 10512 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.315698 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.313946 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:02.318703 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.370736 10579 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.312000 15980 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:02.292009 15914 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:02.384511 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.378433 10581 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:02.370759 16006 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:12:02.386401 10508 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:02.395504 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.396368 10336 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:02.396407 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.397354 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:02.400731 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 155091us
I20260709 12:12:02.401553 10583 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:02.396477 10585 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:02.318616 15801 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:12:02.408962 10582 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:02.399847 15976 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:02.423233 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.423846 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.399634 15838 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.427603 10507 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:12:02.432461 10512 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:02.433789 10504 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:02.435814 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.436497 10508 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:12:02.395504 15762 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:02.438832 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.439428 15712 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.439558 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.447626 10585 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:02.440243 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58559us
I20260709 12:12:02.448120 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.448797 11178 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:02.439895 10513 raft_consensus.cc:2955] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 4 to 10, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:02.441680 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.449934 10587 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:02.443908 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.444973 15801 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:02.376150 15921 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.451438 10511 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:02.452368 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.423125 15976 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.463042 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.466143 10583 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:02.472137 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.467697 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.477512 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.477649 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.479465 10512 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:02.455947 15801 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:02.484838 16038 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:12:02.485881 15801 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.478197 10586 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:02.488191 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.488663 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.489377 10508 raft_consensus.cc:2393] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 12: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:02.481858 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.490525 10507 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.490659 10511 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:12:02.491973 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.467939 15976 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.484825 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.425868 15883 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.484404 10580 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:02.500152 10582 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:02.495412 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.503261 10508 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:02.504535 10508 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:02.505257 10506 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:02.506214 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.507426 10512 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:02.452961 15838 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.514259 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.514972 10586 raft_consensus.cc:2376] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16. Re-sending same reply.
I20260709 12:12:02.503475 10336 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 14, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:02.540705 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.540495 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.498893 16034 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.477281 15762 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.541402 10583 raft_consensus.cc:2376] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14. Re-sending same reply.
I20260709 12:12:02.506299 15801 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:02.556039 16037 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.564323 16037 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.543480 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.494346 15712 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.541724 15729 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:02.606735 15729 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.607117 10511 raft_consensus.cc:2410] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:02.607331 15729 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.608139 15712 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.560170 15985 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:02.560777 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 58474.5us
I20260709 12:12:02.573385 15921 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.570379 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.576661 15762 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:02.577142 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.619336 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.580264 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:02.584276 16030 log.cc:927] Time spent T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:12:02.603691 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:02.605150 16025 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:12:02.605999 16033 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:12:02.543617 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.568358 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.613771 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.624449 10587 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:12:02.568727 10582 raft_consensus.cc:2376] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16. Re-sending same reply.
I20260709 12:12:02.612303 10585 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
W20260709 12:12:02.568689 16028 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:12:02.545606 10581 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
W20260709 12:12:02.622478 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 164213us
W20260709 12:12:02.613911 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104775us
W20260709 12:12:02.569061 15915 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
I20260709 12:12:02.652297 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:02.652691 16037 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:02.613461 16029 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.196s	user 0.001s	sys 0.001s
I20260709 12:12:02.654227 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:02.654736 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:02.655354 16026 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:12:02.655712 10585 raft_consensus.cc:2410] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:12:02.655983 15913 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:02.656215 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.656958 10582 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:02.657722 15913 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:02.658388 15913 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:02.658531 15995 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:02.628558 10583 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:02.651360 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:02.652204 15915 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.612257 15914 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.643374 15838 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.684404 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:02.685604 11212 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 17: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:02.650223 16000 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:02.662257 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:02.663401 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:02.690003 10587 raft_consensus.cc:2376] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14. Re-sending same reply.
W20260709 12:12:02.666441 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.667766 10552 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:02.609535 15729 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:02.559839 10336 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 6 to 13, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:02.659080 15995 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.700011 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:02.682801 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.688073 11217 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:02.696275 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.699435 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:02.682920 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 313096us
I20260709 12:12:02.703629 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:02.706566 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:02.710490 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:02.711844 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.716485 10438 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:02.717931 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:02.718652 10438 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:02.718708 10430 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:02.569267 16034 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.717931 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.721689 10432 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:02.721871 15687 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
W20260709 12:12:02.722651 10403 connection.cc:256] RPC call timeout handler was delayed by -1.4607960200000889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
W20260709 12:12:02.729936 10403 connection.cc:256] RPC call timeout handler was delayed by -1.228411049999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 12:12:02.730352 10403 connection.cc:256] RPC call timeout handler was delayed by -1.1789110499998969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 12:12:02.627391 15985 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.723440 10435 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:02.705569 15995 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:02.725075 11221 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:02.683995 15741 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.712905 10578 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
W20260709 12:12:02.732431 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.732666 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.723448 10336 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 10, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 10 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:02.736557 10403 connection.cc:256] RPC call timeout handler was delayed by -1.1753110499998911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
W20260709 12:12:02.737852 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.707214 15914 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:02.747370 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:02.751952 16034 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.753870 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:02.706710 15711 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:12:02.760823 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 54361.6us
I20260709 12:12:02.757062 15915 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:02.757900 16000 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:02.757795 16049 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:02.736141 15778 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:02.761111 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:02.761654 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51274.1us
I20260709 12:12:02.799052 11213 raft_consensus.cc:2410] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:12:02.762207 15980 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:02.762946 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 59398.9us
I20260709 12:12:02.764861 10507 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:02.765370 16036 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:02.712064 16039 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:02.756192 10336 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 1 to 17, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 17 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.800925 16049 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.802402 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:02.813201 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:02.807361 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:02.808370 10438 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:02.761168 10437 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:02.819694 10336 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 9 to 11, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 11 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.822517 16048 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:12:02.824091 10435 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:02.801215 15980 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.825156 15980 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:02.828214 10431 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:02.828639 15976 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.801569 15914 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:02.833909 15914 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.770356 15741 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.835426 10584 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.837206 15741 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.739946 15687 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:02.842509 15687 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:02.843236 15886 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:02.843101 15913 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.845238 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:02.846114 10513 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:02.818471 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:02.858250 15976 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.808036 15778 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.814466 16049 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:02.819069 15912 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:02.867480 15912 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.867894 15912 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:02.868255 15687 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.868875 15687 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.873322 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:02.816233 16000 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:02.879477 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66359.6us
I20260709 12:12:02.880513 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:02.839243 15985 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:02.891467 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47130.9us
I20260709 12:12:02.871193 16005 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:02.903017 15921 raft_consensus.cc:2764] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:12:02.911634 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:02.891189 16039 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:02.902971 15980 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:12:02.924394 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43973.9us
I20260709 12:12:02.936488 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:02.935698 10336 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 8 to 10, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 10 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:12:02.941653 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 81101.2us
I20260709 12:12:02.892670 15741 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:02.890636 15687 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:02.949918 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:02.958714 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.934724 16034 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:02.948606 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:02.965576 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.962610 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 143478us
I20260709 12:12:02.963584 16039 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:12:02.972011 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.966770 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:02.952842 10336 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:02.967928 16000 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.964648 16034 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:02.973580 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.963779 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:02.971428 15711 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:02.980842 15687 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:02.981019 15711 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.981650 15711 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:02.952163 16005 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:12:02.978900 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 225178us
I20260709 12:12:02.975306 10336 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 7, term changed from 11 to 14, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to <none>, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 14 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:12:02.980863 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.987279 16040 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:02.995692 16040 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:02.996681 16047 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:02.996760 16040 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:02.997468 16047 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:02.998888 16047 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:03.004456 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.979071 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.020108 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.013957 10336 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:02.987771 15913 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:12:03.026341 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:03.028720 16036 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:12:03.030128 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.033094 10479 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.005478 15699 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:12:02.986642 15914 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:12:03.017481 15687 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:12:03.047215 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 59094.7us
I20260709 12:12:02.972970 16039 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.052678 10336 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 8 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:03.056507 15711 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:03.059650 11178 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.627s (SENT)
W20260709 12:12:03.066489 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 99870.9us
W20260709 12:12:03.070739 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.046762 15741 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:03.078187 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:03.048276 15729 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.081440 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.058262 15838 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:03.023494 16049 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.049752 15912 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.068527 15847 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 12:12:03.112174 16059 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:12:03.114495 10513 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:03.115465 15912 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.082645 15687 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:12:03.116746 15687 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:03.117923 15687 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:03.118276 15687 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.108912 15980 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.120155 15883 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:03.120606 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:03.122359 10504 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:03.123621 15914 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:03.124173 15914 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.125402 15914 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:03.128264 16057 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:12:03.129117 16049 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.109277 16001 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.046134 16034 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:03.139675 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 90413.4us
I20260709 12:12:03.091925 16036 raft_consensus.cc:2764] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:12:03.087723 15741 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.087654 15838 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.126801 15914 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:03.174451 10512 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:03.177511 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.179010 10504 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:03.181089 10511 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:03.183681 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.088677 15729 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.183985 15687 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:03.187395 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:03.188211 10509 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:03.188263 16036 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:03.188979 16036 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.191320 16049 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:03.191704 15980 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:03.192778 16067 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:03.179967 15838 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.193248 16067 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.193898 15976 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:03.194844 15976 consensus_queue.cc:1059] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:03.195607 15976 consensus_queue.cc:1243] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:12:03.196287 15976 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:03.194340 16067 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.197111 15976 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:03.197504 15976 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:03.198004 16034 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:03.198369 16047 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:12:03.195142 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.199640 16041 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:03.199718 15741 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.200266 16041 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.200829 15729 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.198401 15915 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:03.201617 15915 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:03.202584 15915 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.198925 16047 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:03.203972 15915 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:03.204210 16047 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.205657 16047 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:03.210300 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:03.209589 16041 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:03.211676 16036 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:03.215407 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:03.223977 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:03.225072 10513 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:03.235380 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61636.2us
W20260709 12:12:03.247989 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74863.6us
I20260709 12:12:03.247529 16041 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.255528 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:03.257205 10584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:03.257596 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:03.257952 10584 raft_consensus.cc:606] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.258236 11217 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:03.258368 10584 raft_consensus.cc:614] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:03.265156 15778 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:03.266158 16042 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:12:03.141994 16001 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.270184 16001 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:03.300496 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.309103 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:03.317306 11217 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:03.318753 11221 raft_consensus.cc:1240] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:03.336190 10513 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
W20260709 12:12:03.317337 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 235980us
W20260709 12:12:03.329986 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 245491us
I20260709 12:12:03.340970 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:03.341233 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.314049 15711 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:03.254884 16041 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.342669 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:03.318612 16055 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.350133 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.354463 10509 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
W20260709 12:12:03.354456 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70917us
I20260709 12:12:03.356496 10552 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:03.357620 16048 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:03.358210 16048 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:03.359318 16048 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.361037 16072 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:03.363245 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:03.365887 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:03.311996 15976 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.351377 15915 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:03.379371 15778 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:03.379973 15778 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.380391 15778 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.380676 16041 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:03.381829 15995 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
I20260709 12:12:03.366642 16055 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.385308 16055 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:03.386411 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:03.387204 15985 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:03.392859 16067 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:03.396765 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.382337 15995 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.402730 15995 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:03.403729 15985 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:03.407672 15985 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.408181 15985 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:03.412703 15687 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:03.405810 16041 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:03.415900 16041 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:03.416369 16041 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:03.417548 16041 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.417981 15838 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.420567 15838 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:03.421375 16038 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.421978 16038 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:03.419512 16039 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.424471 16041 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.370334 15921 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:03.425184 15921 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.425232 15915 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:03.425702 15921 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:03.426517 15915 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:03.428503 15712 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:03.429024 15712 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.424543 16038 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:03.406795 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:03.431577 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.447813 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 97739us
W20260709 12:12:03.449518 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 108339us
I20260709 12:12:03.430205 15712 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:03.451059 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 173454us
I20260709 12:12:03.468273 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:03.473824 10583 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:12:03.493219 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:03.475122 15838 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:03.472462 15976 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:03.481953 16036 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:03.498175 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.498842 16074 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:03.503232 15914 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:03.511439 16039 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:03.517198 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.528090 11216 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:03.528759 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.531334 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.535632 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:03.527562 16039 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.537149 16039 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:03.538450 16047 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:03.539139 16047 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.543286 10506 raft_consensus.cc:2393] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:03.496239 15838 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.523795 16036 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:03.550827 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55621.3us
W20260709 12:12:03.554486 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.555109 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:03.555814 10512 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.556325 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:03.557201 11217 raft_consensus.cc:2410] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:12:03.558761 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:03.530367 15712 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:03.564617 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:03.564481 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:03.565311 11212 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:03.565479 10431 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:03.563665 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:03.533133 16071 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:03.564466 10478 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.566813 10435 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:03.563799 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:03.567648 10436 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:03.568611 10437 raft_consensus.cc:2410] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:12:03.568545 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:03.532328 16068 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:03.573992 10432 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:03.575730 11221 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:03.547451 15995 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:03.509358 15921 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.557269 15838 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:03.596006 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.536s (SENT)
I20260709 12:12:03.589639 11217 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:12:03.599562 11212 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:12:03.596660 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.595196 10510 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:03.559900 16036 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:03.541883 15847 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:12:03.577282 16068 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:03.614830 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:03.619095 11212 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:12:03.619642 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.616087 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:03.576326 16045 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:03.621115 11217 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:03.587812 15995 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.637051 11221 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:03.637285 11217 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:03.644958 10432 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:03.619860 16068 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:03.623075 15980 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:03.601679 16034 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:12:03.622690 16071 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.625982 15995 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:03.673177 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 97890us
I20260709 12:12:03.674147 10512 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
W20260709 12:12:03.678757 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.645656 16045 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:03.660111 16065 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.687141 15699 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:03.686708 15778 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.692842 15778 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:03.695220 16047 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.651373 15915 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:12:03.707314 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:03.714609 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.716392 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:03.717209 10432 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.730921 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:03.705997 15699 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:03.741647 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:03.743002 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 193390us
W20260709 12:12:03.750046 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 185680us
I20260709 12:12:03.746369 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:03.700251 16065 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.746870 16055 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:03.751904 16048 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:12:03.753700 15976 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:03.773566 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.761967 10432 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:03.774811 10431 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:03.771473 16047 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.765507 15921 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:03.782968 16040 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.807631 16040 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:03.815040 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.810868 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:12:03.825192 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.855036 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.764487 16091 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:12:03.796360 16094 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:03.874970 10583 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:12:03.763933 16071 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.765048 15699 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.775727 16045 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.857801 15921 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:03.806792 16047 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:03.920462 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 174183us
I20260709 12:12:03.824854 16040 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:03.867758 16048 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:03.885811 16097 log.cc:927] Time spent T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:12:03.916487 16072 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.001s
I20260709 12:12:03.919867 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:03.932039 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 121234us
I20260709 12:12:03.935429 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.962798 10583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:03.963791 10583 raft_consensus.cc:606] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.964267 10583 raft_consensus.cc:614] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:03.936853 16072 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:03.970917 16095 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:03.971086 16093 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:03.973337 11221 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:12:03.990468 15985 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.003s
W20260709 12:12:03.991434 16089 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 12:12:03.992014 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.026s (SENT)
I20260709 12:12:03.996459 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:03.942343 16055 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:03.950629 16065 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:04.005421 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67372us
I20260709 12:12:03.931258 16038 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:12:03.935774 16048 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:03.928138 15921 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.014367 15838 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.023343 15838 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.028692 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:04.004459 16080 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:04.037873 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:04.045125 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:04.050922 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:04.051841 15838 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:04.052520 15838 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:04.053623 15838 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.037365 15883 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:04.035563 15914 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:04.056728 15914 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.057480 15914 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:04.041705 16074 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
W20260709 12:12:04.062373 16039 consensus_meta.cc:287] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.005s
I20260709 12:12:04.062999 16039 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.063450 16055 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.055014 16102 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:12:04.071985 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:04.056804 15883 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:04.027608 16095 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:04.039037 16088 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.268s	user 0.002s	sys 0.000s
I20260709 12:12:04.011516 15913 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:12:04.012159 15985 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.120628 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:04.122238 10512 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:12:04.041558 16094 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:04.124636 10581 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:04.075373 15883 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:04.125830 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:04.126837 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:04.083169 16102 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.077348 16047 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:04.121141 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.138779 10583 raft_consensus.cc:2410] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:12:04.083469 15921 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:04.140936 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:04.142588 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.143275 10587 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:12:04.145355 16072 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:04.146132 16039 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.146512 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:04.147166 16094 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.147945 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.148567 10586 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.149032 10579 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:12:04.149518 16113 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:12:04.150241 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.147647 10512 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:04.151643 10585 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:04.152793 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:04.152810 16110 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:12:04.153345 10585 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.155880 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.156781 10581 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:12:04.079841 16095 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:04.160974 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.161600 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 169653us
I20260709 12:12:04.161758 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.163328 10582 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:04.150851 10579 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:04.159282 10578 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:12:04.169816 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48801.3us
I20260709 12:12:04.170526 10580 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:04.170679 10587 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:04.171844 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:04.074930 16036 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.182029 10479 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:04.184378 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.185108 10584 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.187323 10587 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:04.187757 10478 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:04.189188 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:04.195369 10585 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:04.197276 10586 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:04.159770 15980 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:12:04.131999 15712 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:12:04.200307 11213 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:04.187144 15838 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:04.200241 16082 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:04.203078 15838 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:04.204056 15985 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.204759 15883 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.204080 15838 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:04.214340 16067 consensus_meta.cc:287] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.000s
W20260709 12:12:04.221735 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.138358 15913 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:12:04.228080 10511 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:04.229106 10579 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
W20260709 12:12:04.223505 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 178518us
I20260709 12:12:04.224967 10586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:04.227342 11213 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:12:04.232347 10586 raft_consensus.cc:606] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.232898 10586 raft_consensus.cc:614] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:04.240545 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.247517 10584 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:04.232295 11213 pending_rounds.cc:85] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 9
I20260709 12:12:04.265172 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:04.220502 16074 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:04.272181 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:04.277074 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:04.277812 10580 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:04.267218 15995 consensus_meta.cc:287] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.568s	user 0.015s	sys 0.000s
I20260709 12:12:04.238386 16067 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:04.238533 16082 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:04.251783 16114 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.265202 11213 pending_rounds.cc:107] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.10
I20260709 12:12:04.169911 16102 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:04.293339 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 331610us
I20260709 12:12:04.296797 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.297868 16091 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:04.298300 10437 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:12:04.307426 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.310458 11213 raft_consensus.cc:2917] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Aborting config change with OpId 14.10: Aborted: Op aborted by new leader
I20260709 12:12:04.321861 11213 raft_consensus.cc:2889] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:04.316274 10431 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:04.323865 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.317606 10438 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:12:04.327708 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.331768 10429 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:12:04.327451 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.343860 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.344679 10433 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.346352 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:04.290871 16074 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:04.362550 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:04.373433 10553 leader_election.cc:336] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.557s (SENT)
I20260709 12:12:04.374192 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.363492 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:12:04.376976 16111 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
W20260709 12:12:04.345059 15976 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.004s
I20260709 12:12:04.379547 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.384742 10438 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:04.383268 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.384851 10429 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:12:04.328102 16041 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:04.291263 15995 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.392910 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:04.406899 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.413707 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.326113 16123 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:04.416404 10434 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:04.419605 10431 raft_consensus.cc:2410] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:12:04.424721 10553 leader_election.cc:336] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.561s (SENT)
I20260709 12:12:04.424769 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:04.426273 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 238591us
I20260709 12:12:04.427560 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:04.319475 16114 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.389657 15976 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:04.388044 15980 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.367213 16073 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 12:12:04.433794 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.345894 16000 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:12:04.395756 16041 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:04.421172 16055 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:04.454250 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.450950 15980 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.423715 16123 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.432171 16047 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.434763 15699 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:04.502139 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.499482 16093 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:04.508056 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 119799us
I20260709 12:12:04.436744 15913 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:12:04.431764 16082 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.439741 16071 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:12:04.551805 15980 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:04.553786 10434 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:04.555042 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.555692 10434 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:04.555965 16127 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:12:04.556641 16102 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:04.557682 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.505254 16036 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.558694 16114 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:04.558959 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.504695 16074 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.559605 10435 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:04.560670 15995 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.503355 16047 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.565613 16047 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:04.483395 16123 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:04.568948 16068 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.892s	user 0.013s	sys 0.004s
I20260709 12:12:04.507750 16093 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:04.532512 16124 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:12:04.533201 16125 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:12:04.534324 16122 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:12:04.535386 16128 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:12:04.539670 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.543257 16041 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.503261 15711 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:04.547349 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:04.588093 16036 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.529670 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.574278 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:04.624559 10433 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:04.574478 10437 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:04.455281 15741 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:12:04.587932 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.595703 10435 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:12:04.606312 16093 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.603031 16068 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:04.611514 10431 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:04.547756 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.600452 16047 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.628305 10429 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:04.628928 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.628643 16036 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:04.613971 15914 consensus_meta.cc:287] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.556s	user 0.016s	sys 0.000s
I20260709 12:12:04.633955 10438 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:04.634384 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.637535 10437 raft_consensus.cc:2376] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15. Re-sending same reply.
I20260709 12:12:04.637900 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.634387 10434 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:12:04.636220 10553 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.788s (SENT)
I20260709 12:12:04.640424 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.640668 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.639752 10435 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:12:04.641595 10433 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:04.646411 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.650333 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.664181 10432 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:04.647485 10431 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:04.661619 10510 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:04.672737 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.673494 10436 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:04.677075 10505 raft_consensus.cc:606] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.677145 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.680974 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:04.681788 11215 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:04.665535 15711 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.661511 16114 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:04.640048 16047 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.689788 16036 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.681809 15847 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:04.692624 16068 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.641403 15914 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:04.693935 16136 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:12:04.695158 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.697868 10580 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:12:04.707628 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:04.708460 16040 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:04.714366 16040 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:04.716518 16048 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:04.721647 16041 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:04.715471 16040 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.728673 15914 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:04.703666 15847 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:04.729652 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.731153 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.736157 16119 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:04.737147 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.737701 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.694645 15711 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:04.748212 16108 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.690s	user 0.007s	sys 0.000s
W20260709 12:12:04.749830 16139 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 12:12:04.755121 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.758888 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.718827 16047 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:04.789649 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.790866 10553 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.974s (SENT)
I20260709 12:12:04.791448 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:04.793363 11178 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.644s (SENT)
I20260709 12:12:04.805027 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:04.810629 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.817565 15711 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.807229 16047 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:12:04.845431 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.764802 16119 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.851037 16119 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:04.853137 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:04.870615 10582 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:04.871807 16038 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.872586 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.873461 10437 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:12:04.857206 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66397.1us
I20260709 12:12:04.859659 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.875597 10433 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:04.868964 15847 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.891492 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.892347 10437 raft_consensus.cc:2376] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16. Re-sending same reply.
I20260709 12:12:04.893471 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:04.899264 10553 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.991s (SENT)
I20260709 12:12:04.916683 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.894750 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.938328 10434 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:12:04.920961 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.907563 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.942947 10436 raft_consensus.cc:2410] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:12:04.904973 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2623) took 1156 ms. Trace:
I20260709 12:12:04.915512 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:04.946581 11212 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:04.947872 10435 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:12:04.948253 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:04.949141 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.949196 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.943605 10510 rpcz_store.cc:276] 0709 12:12:03.748095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.748275 (+   180us) service_pool.cc:224] Handling call
0709 12:12:03.748698 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:04.668895 (+920197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:04.904660 (+235765us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b5400413054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:04.904892 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071999744}
W20260709 12:12:04.924167 16132 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.380s	user 0.003s	sys 0.000s
I20260709 12:12:04.956418 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.957645 10435 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:04.957532 16116 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:04.958837 16123 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:04.959304 16116 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:04.961086 16134 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
I20260709 12:12:04.961720 16123 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:04.962941 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.963366 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:04.963995 10438 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:04.958848  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:04.964017 10433 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:04.966868 16140 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.190s	user 0.002s	sys 0.000s
I20260709 12:12:04.969357 10429 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:04.965514 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.971304 16055 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:04.972002 10434 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:04.973245 10434 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:04.958839 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.975283 15976 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:04.975936 10435 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:04.974852 10434 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:04.976438 10438 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:04.976719 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.987249 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:04.992484 10429 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:04.993144 10513 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:04.988922 10431 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:12:04.971213 16130 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.368s	user 0.000s	sys 0.003s
I20260709 12:12:04.995412 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:04.995723 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:04.996165 11216 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:04.997707  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:04.995473 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:04.999689 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:04.898912 15847 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:04.926693 16141 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.180s	user 0.002s	sys 0.000s
I20260709 12:12:04.999183  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:04.872450 16038 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:05.005829 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.020295 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.024080 10436 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:12:05.024191 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:05.027412 10433 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:05.033614 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.034602 11213 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:12:05.036022 10580 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:12:05.036756 10433 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:12:05.037154 10433 pending_rounds.cc:85] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:12:05.037408 10433 pending_rounds.cc:107] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:12:05.037707 10433 raft_consensus.cc:2889] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:05.038793 11216 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:05.039304 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:05.033764 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:05.040093 10583 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:05.005501 15976 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:05.032750 16045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.026s	sys 0.026s
I20260709 12:12:05.043612 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.045460 10586 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:05.046545 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.047257 10429 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:12:05.047688 10578 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:12:05.048127 15838 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:05.049762 15913 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:05.050499 15913 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:05.051787 15913 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.053280 15838 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:05.054409 15838 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.055505 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.056169 10435 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:05.044314 10584 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:05.061200 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.062252 10437 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:05.066644 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.087457 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.093256 10579 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:12:05.093722 10579 pending_rounds.cc:85] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 7
I20260709 12:12:05.093957 10579 pending_rounds.cc:107] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:12:05.094189 10579 raft_consensus.cc:2889] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:05.066718 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.042968 16045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.026s	sys 0.026s
I20260709 12:12:05.105748 10584 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:12:05.107873 10435 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:05.108268 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 710) took 1133 ms. Trace:
I20260709 12:12:05.034664 16116 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.116729 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:05.040033 16114 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:05.108844 11217 rpcz_store.cc:276] 0709 12:12:03.969079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.976116 (+  7037us) service_pool.cc:224] Handling call
0709 12:12:04.014953 (+ 38837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:04.138089 (+123136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:04.138129 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:04.165528 (+ 27399us) log.cc:844] Serialized 26 byte log entry
0709 12:12:04.449848 (+284320us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:04.449892 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:04.449961 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:04.970197 (+520236us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:05.088087 (+117890us) raft_consensus.cc:1685] finished
0709 12:12:05.088121 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.102501 (+ 14380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":780,"raft.queue_time_us":40474,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":20509952,"thread_start_us":311742,"threads_started":2,"wal-append.queue_time_us":367913}
I20260709 12:12:05.129124 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.132560 10579 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:12:05.130096 10578 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:12:05.140707 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 203062us
I20260709 12:12:05.031504 16072 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:12:05.105193 16102 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.167264 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.168323 10436 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:12:05.129264 16073 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.181308 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.190201 10437 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:05.088526 15976 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:05.212172 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.213346 10513 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:05.215044 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:05.215822 10513 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:05.195716 16082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.000s	sys 0.033s
I20260709 12:12:05.165295 16138 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:12:05.222708 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.231498 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.165691 16000 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:12:05.231293 10582 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:05.239799 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:05.176249 16102 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:12:05.232172 16082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.001s	sys 0.033s
I20260709 12:12:05.248822 10430 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:05.200583 16073 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.246522 16114 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:12:05.271169 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.273340 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.274466 10437 raft_consensus.cc:2368] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19.
I20260709 12:12:05.291420 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.249385 16117 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.293860 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.302531 10437 raft_consensus.cc:2376] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10. Re-sending same reply.
I20260709 12:12:05.296778 10513 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
W20260709 12:12:05.220650 16074 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.002s	sys 0.020s
I20260709 12:12:05.292303 10430 raft_consensus.cc:2376] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17. Re-sending same reply.
I20260709 12:12:05.273480 16114 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:05.310479 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.306758 16142 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
W20260709 12:12:05.318282 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:05.313840 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72640.3us
I20260709 12:12:05.316076 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.308180 16074 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.003s	sys 0.021s
I20260709 12:12:05.293008 15699 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:05.320662 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:05.330067 15913 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:05.330665 15913 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.331204 15913 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:05.330732 16155 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:12:05.336126 16145 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:12:05.338021 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 713) took 1129 ms. Trace:
I20260709 12:12:05.338578 11214 rpcz_store.cc:276] 0709 12:12:04.208337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.208999 (+   662us) service_pool.cc:224] Handling call
0709 12:12:04.209358 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:04.209775 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:04.209804 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:04.755652 (+545848us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b2c0001cb68. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:12:04.756246 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:12:05.022190 (+265944us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:05.022234 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.022307 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:05.337553 (+315246us) raft_consensus.cc:1685] finished
0709 12:12:05.337585 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.337907 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1869,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":52,"spinlock_wait_cycles":23288064,"thread_start_us":265398,"threads_started":1,"wal-append.queue_time_us":431816}
I20260709 12:12:05.317502 16038 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.360152 15699 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:05.364847 16093 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:05.369550 15913 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.382028 15913 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.383057 16073 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.293165 16067 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:12:05.384689 16114 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.391445 11215 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 16. Ops: []
W20260709 12:12:05.338239 16146 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:12:05.400887 15976 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:05.404803 16102 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:12:05.311630 16117 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.409134 16117 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:05.413888 16144 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:12:05.361862 15699 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.426712 10583 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
W20260709 12:12:05.342680 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 467815us
W20260709 12:12:05.439133 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 215509us
I20260709 12:12:05.443048 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.458326 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.458819 10438 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
W20260709 12:12:05.456405 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:05.460568 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.477901 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.479106 10431 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 13: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:05.480157 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:05.480568 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 382641us
I20260709 12:12:05.481078 11221 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:05.485219 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:05.488412 10431 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:05.494156 10553 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.048s (SENT)
I20260709 12:12:05.494932 11213 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:12:05.494866 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:05.500268 10438 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:05.500748 10438 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:05.511111 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.499786 15980 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:05.524268 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 427965us
I20260709 12:12:05.532016 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.533174 10437 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:05.528611 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:05.522051 10438 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:05.534263 10431 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:12:05.535524 10431 pending_rounds.cc:85] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 9
W20260709 12:12:05.528712 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44842.5us
I20260709 12:12:05.536573 10431 pending_rounds.cc:107] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.10
I20260709 12:12:05.536445 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.537047 10431 raft_consensus.cc:2917] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Aborting config change with OpId 14.10: Aborted: Op aborted by new leader
I20260709 12:12:05.537532 10431 raft_consensus.cc:2889] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:05.537413 10553 leader_election.cc:336] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.048s (SENT)
I20260709 12:12:05.537650 10435 raft_consensus.cc:2410] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 12:12:05.541776 16115 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.009s
W20260709 12:12:05.542400 16115 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.000s	sys 0.010s
I20260709 12:12:05.526443 16170 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:05.552489 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.527212 15980 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.557032 10583 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:05.557456 10436 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:05.559048 10478 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:05.560192 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:05.560762 16067 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.561398 16067 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.563814 16067 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.571170 15980 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:05.572458 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.573487 10432 raft_consensus.cc:2410] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:12:05.539839 16073 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.560894 10583 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:05.580041 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.580817 10580 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:05.574479 16073 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:05.588192 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50925.7us
I20260709 12:12:05.589927 10436 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:05.595847 10436 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:05.589833 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:05.596151 10553 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.064s (SENT)
W20260709 12:12:05.594379 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.596848 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:05.596819 10586 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:05.596921 10582 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:12:05.597415 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:05.599601 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.599596 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.587545 15847 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.003s
W20260709 12:12:05.601401 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 73037us
I20260709 12:12:05.602548 10432 raft_consensus.cc:2410] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:12:05.606623 10431 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:05.576737 15699 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:05.608954 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.612059 10512 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:12:05.612495 10512 pending_rounds.cc:85] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 11
I20260709 12:12:05.612830 10512 pending_rounds.cc:107] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:12:05.612587 10583 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:05.613173 10512 raft_consensus.cc:2889] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:05.609748 10437 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:05.606266 15914 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:12:05.610889 16170 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:05.608791 15913 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:05.618423 16166 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:05.618952 15913 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:05.619654 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60796.4us
I20260709 12:12:05.621505 16172 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:05.621624 15913 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.622038 16172 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.622473 16172 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:05.623675 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.624249 10438 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:05.624432 10511 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:05.625617 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:05.627183 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.614220 10436 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.632200 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.626783 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:05.636180 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.643299 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.643563 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.648506 10437 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:05.649799 10436 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:05.618968 16166 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.642577 16117 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:05.658636 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:05.661892 10437 raft_consensus.cc:2393] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:05.664593 10438 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:12:05.623817 15699 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:05.669409 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.672401 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.675880 10436 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:12:05.676343 10436 pending_rounds.cc:85] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 6
I20260709 12:12:05.660483 16117 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.677379 10436 pending_rounds.cc:107] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:12:05.651314 15847 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:12:05.654943 16166 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:05.696511 16073 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:05.697243 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:05.677454 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56670.7us
I20260709 12:12:05.702356 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.703327 10438 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:05.705148 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:05.705413 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.706023 10511 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:05.708494 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.678890 10436 raft_consensus.cc:2889] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:05.701344 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 68030.6us
I20260709 12:12:05.691790 16117 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.705893 10432 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:05.651554 16171 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:12:05.687624 15699 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:05.712709 10504 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:05.630892 16176 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:12:05.715034 10508 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:12:05.720264 10508 pending_rounds.cc:85] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 11
I20260709 12:12:05.720582 10508 pending_rounds.cc:107] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:12:05.720849 10508 raft_consensus.cc:2889] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:05.719723 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:05.715950 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.719439 10509 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:12:05.715250 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:12:05.721865 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 96365us
I20260709 12:12:05.723555 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.724886 10504 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16.
W20260709 12:12:05.710661 10553 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.158s (SENT)
I20260709 12:12:05.711508 16114 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:05.726279 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:05.717995 16171 consensus_queue.cc:1243] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:12:05.727888 11178 leader_election.cc:336] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.211s (SENT)
I20260709 12:12:05.731426 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:05.728629 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.735489 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.735821 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.748729 16115 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:05.759279 11178 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.240s (SENT)
I20260709 12:12:05.736590 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:05.781464 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 70861.7us
W20260709 12:12:05.783030 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.214s (SENT)
I20260709 12:12:05.779987 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:05.746587 15883 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:05.787400 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:05.747366 16121 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:05.790009 16040 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:05.804438 16040 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:05.804993 16040 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:05.806049 16040 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:05.807073 16121 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:05.809718 16121 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.799160 16115 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:05.846232 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66356.5us
W20260709 12:12:05.878058 10553 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.501s (SENT)
I20260709 12:12:05.881098 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:05.890829 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 886) took 1162 ms. Trace:
I20260709 12:12:05.877696 15980 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:05.941704 11215 rpcz_store.cc:276] 0709 12:12:04.728654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.765405 (+ 36751us) service_pool.cc:224] Handling call
0709 12:12:05.391125 (+625720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.399786 (+  8661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.890446 (+490660us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400606150. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:05.890726 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374829312}
I20260709 12:12:05.943711 15980 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:05.870673 16115 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:12:05.909224 16172 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:05.947377 16172 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:05.914312 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.949426 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.950603 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:05.923820 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.964917 10587 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:12:05.926762 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:05.928146 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.929639 10579 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:05.930769 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.968463 10581 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:05.932297 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.933399 16080 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:05.933243 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:05.939224 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 228813us
I20260709 12:12:05.976599 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.967048 10578 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:05.940018 15699 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:12:05.940909 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 218349us
I20260709 12:12:05.955340 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:05.961141 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:05.976639 10584 raft_consensus.cc:2410] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:12:05.961485 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.983477 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.984974 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:05.959182 16158 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.973882 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.921581 10583 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:12:05.986555 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.987354 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.987982 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:05.989701 10512 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:05.994184 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.994704 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:05.995155 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117143us
I20260709 12:12:05.995599 10510 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:05.995734 16115 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:12:05.996268 10553 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.617s (SENT)
I20260709 12:12:05.993777 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:05.989068 10478 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:06.004972 10508 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:12:06.011550 10553 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.639s (SENT)
I20260709 12:12:06.011332 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:06.012233 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.012734 10509 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:05.992776 16166 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.019477 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:06.008464 16080 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:06.000730 16182 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
W20260709 12:12:06.053900 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.064148 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.997697 15913 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:12:06.067487 10507 raft_consensus.cc:1217] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[11.7-11.7]   Dedup: 11.7->[]
W20260709 12:12:06.063251 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51738.2us
I20260709 12:12:06.068046 10507 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:12:06.079527 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.092841 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.076910 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.094687 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.095091 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.007148 10510 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:12:06.009614 16072 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:06.094792 10505 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:06.064921 10581 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
W20260709 12:12:06.064774 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55568.6us
I20260709 12:12:06.097864 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:06.098610 10512 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:12:06.111960 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:05.998064 16158 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.115015 10431 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:06.118777 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:06.067706 16182 consensus_queue.cc:1243] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 52bae2d2caf94d97b1049ecd4f82a2df log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:12:06.119793 10436 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:06.123986 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:06.126120 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:06.062781 16073 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.136981 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.137423 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:06.072865 16115 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:12:06.148886 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.152024 10403 connection.cc:256] RPC call timeout handler was delayed by -2.5329129700000976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
I20260709 12:12:06.148375 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.164052 10437 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:06.137872 10431 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:06.167562 10511 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: [12.8-13.9]
W20260709 12:12:06.170401 10403 connection.cc:256] RPC call timeout handler was delayed by -2.484312970000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 12:12:06.165943 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65564.8us
I20260709 12:12:06.139400 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:06.170862 10403 connection.cc:256] RPC call timeout handler was delayed by -2.4636129699999856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
I20260709 12:12:06.138885 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:06.146834 10509 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:06.122304 16193 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.109249 15976 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:12:06.161013 16045 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.100550 16072 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:06.189934 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51113.3us
I20260709 12:12:06.165203 16186 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:12:06.171907 10403 connection.cc:256] RPC call timeout handler was delayed by -2.4402129700001751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 12:12:06.171916 16191 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:06.177301 16071 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:06.196600 15712 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:06.208034 16191 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.209188 16115 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:12:06.179415 16080 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.212656 16115 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:06.213023 16191 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:06.214082 15712 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:06.215227 15712 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.213094 16115 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:06.217816 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.220742 10432 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:06.170698 16117 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:06.218236 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 113619us
I20260709 12:12:06.201433 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.173995 16176 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:12:06.163749 16073 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.198364 16186 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.222476 10582 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:06.194964 16072 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:06.198139 10403 connection.cc:256] RPC call timeout handler was delayed by -2.3781129700000747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 12:12:06.231742 10403 connection.cc:256] RPC call timeout handler was delayed by -2.3484129700000267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260709 12:12:06.232143 10403 connection.cc:256] RPC call timeout handler was delayed by -2.3061129699999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
W20260709 12:12:06.232445 10403 connection.cc:256] RPC call timeout handler was delayed by -2.2080129700000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 12:12:06.232731 10403 connection.cc:256] RPC call timeout handler was delayed by -2.0982129700000769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
I20260709 12:12:06.206774 16071 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:06.233072 10403 connection.cc:256] RPC call timeout handler was delayed by -1.9380129700000452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
W20260709 12:12:06.233414 10403 connection.cc:256] RPC call timeout handler was delayed by -1.911912970000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 12:12:06.233698 10403 connection.cc:256] RPC call timeout handler was delayed by -1.9047129699999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
I20260709 12:12:06.235221 10436 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:06.217234 16166 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:06.238291 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.240903 10432 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:06.242486 10578 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.232807 16186 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.236193 16114 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.251346 16160 consensus_queue.cc:1059] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:06.254241 15980 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:06.254642 15980 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:06.242848 15847 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:06.198544 16045 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:06.285342 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51103.2us
I20260709 12:12:06.280921 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:06.237210 16071 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.299930 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.303287 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.191955 16193 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.315460 16191 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:06.316792 10432 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:06.316463 15847 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:12:06.319087 15980 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.292037 16047 raft_consensus.cc:2764] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:12:06.334048 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.307865 16158 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:06.279341 16114 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:06.342922 16179 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.698s	user 0.004s	sys 0.000s
I20260709 12:12:06.343308 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:06.289242 16159 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:12:06.411162 16159 consensus_queue.cc:1243] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:12:06.346288 16117 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:06.347288 16034 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:06.414455 16034 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:06.419845 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 85890.7us
I20260709 12:12:06.421887 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.428236 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:06.429164 10510 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.429986 10504 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:06.251951 16170 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:12:06.341090 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.363619 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.366750 11214 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.367369 10438 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:06.368909 16071 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:06.332039 16115 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.384547 10552 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:06.444185 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.386029 16073 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:06.388667 16186 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:06.446549 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 62157.4us
I20260709 12:12:06.401793 16193 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:06.404038 16045 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:06.405220 16114 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:06.408587 15980 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:06.357865 10436 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:12:06.437263 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73750.5us
I20260709 12:12:06.406848 16055 raft_consensus.cc:1064] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:06.436185 16116 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:06.450655 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 109693us
I20260709 12:12:06.463680 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.452349 10583 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:12:06.454363 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:06.457366 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:06.449761 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:06.460081 10581 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:06.461333 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:06.466435 10438 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:06.466465 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:06.467077 10579 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:06.449365 16123 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:06.470695 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.471807 10436 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:12:06.484699 10436 pending_rounds.cc:85] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 6
I20260709 12:12:06.484995 10436 pending_rounds.cc:107] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:12:06.485327 10436 raft_consensus.cc:2889] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.450001 16047 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:06.458580 16000 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:06.493021 10583 pending_rounds.cc:85] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 6
I20260709 12:12:06.499097 10583 pending_rounds.cc:107] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:12:06.499473 10583 raft_consensus.cc:2889] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.456965 16193 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:06.454334 16119 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:06.498328 10580 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:12:06.501911 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:06.512797 10580 pending_rounds.cc:85] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 7
I20260709 12:12:06.509218 10438 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:06.513106 10580 pending_rounds.cc:107] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:12:06.513389 10580 raft_consensus.cc:2889] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.460515 16040 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:12:06.520154 10584 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:12:06.516239 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 59472.9us
I20260709 12:12:06.490716 16160 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:12:06.529531 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 107876us
I20260709 12:12:06.484253 16123 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:06.531507 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.469213 16114 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:12:06.469890 16116 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:06.554539 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.537274 16123 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.620692 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.537081 16117 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:06.537242 16000 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:06.628299 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 157689us
I20260709 12:12:06.647054 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.648882 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.539532 16193 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:06.521651 16119 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.661716 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:06.656832 16181 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:12:06.666857 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46300.7us
I20260709 12:12:06.669615 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:06.668639 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:06.670121 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:06.670336 10587 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:06.537921 16034 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:06.704252 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:06.708914 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:06.710702 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:06.725980 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.727999 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.726244 16045 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:12:06.735146 16205 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:12:06.736645 16186 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:06.738363 16119 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:06.691419 16208 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:06.707422 16210 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:06.726254 16072 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.743511 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:06.703311 16170 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.752843 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:06.726469 16100 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:06.747912 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:06.731029 16200 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:06.743726 16211 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.747256 16214 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:12:06.732002 16185 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:12:06.751562 16114 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:06.743606 16071 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
W20260709 12:12:06.825786 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 125183us
I20260709 12:12:06.801230 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.831113 10510 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:06.743338 16216 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:06.739987 16158 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:12:06.797075 15995 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:06.793906 16200 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.744750 16166 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:12:06.743481 15712 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:12:06.750360 16176 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:12:06.857004 10578 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.883787 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:06.744148 16000 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.842087 16216 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.849992 16074 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:06.760624 16170 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:06.858021 15995 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:06.830343 16165 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 12:12:06.937345 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91145us
I20260709 12:12:06.801373 16215 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:12:06.860814 16200 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:06.881116 16073 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:12:06.913713 16216 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:06.960060 16206 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
W20260709 12:12:06.961861 16207 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
W20260709 12:12:06.974392 11178 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.157s (SENT) [suppressed 42 similar messages]
I20260709 12:12:06.832894 16211 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:06.957304 15995 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:06.935838 16101 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:12:06.996143 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:06.998786 10512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.033s	sys 0.041s
W20260709 12:12:07.002038 10512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.034s	sys 0.041s
I20260709 12:12:07.043215 16221 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:07.044183 16221 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.070067 10433 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:07.070225 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.071548 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.072170 10433 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.993991 16208 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.079356 10436 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:12:07.047902 16219 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.106s	user 0.001s	sys 0.001s
I20260709 12:12:07.081459 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:07.099233 11177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.012s	sys 0.007s
I20260709 12:12:07.063136 16185 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:12:07.099742 11177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.012s	sys 0.008s
I20260709 12:12:07.064074 16034 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.113664 10433 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:12:07.110561 10579 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.006s	sys 0.016s
W20260709 12:12:07.122090 10579 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.006s	sys 0.017s
I20260709 12:12:07.008172 16080 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
W20260709 12:12:07.116683 11175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.000s	sys 0.022s
I20260709 12:12:07.085103 16072 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:07.124576 11175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.000s	sys 0.023s
I20260709 12:12:07.130383 10578 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:07.110795 16067 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:07.142828 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 438659us
I20260709 12:12:07.120417 16185 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:07.148181 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:07.149089 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 174758us
I20260709 12:12:07.129613 16221 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:07.151283 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:07.151960 11215 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:07.153450 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.090510 16208 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:07.119954 16034 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.158739 11214 pending_rounds.cc:85] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 14
I20260709 12:12:07.182367 11214 pending_rounds.cc:107] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:12:07.182915 11214 raft_consensus.cc:2889] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:07.159479 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:07.180809 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:07.187296 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:07.187435 16238 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:07.189000 16238 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:07.189483 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:07.189644 16238 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:07.190353 16238 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:07.190738 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:07.147570 16067 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:07.200194 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:07.201596 11217 raft_consensus.cc:2410] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:07.158094 16045 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:07.210634 10583 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.010s
W20260709 12:12:07.213210 10583 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.000s	sys 0.010s
W20260709 12:12:07.278869 16229 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 12:12:07.279440 16227 log.cc:927] Time spent T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 12:12:07.280467 16222 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 12:12:07.283815 16246 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:12:07.286615 16208 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.287556 16208 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.288389 16208 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:07.290647 16234 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:12:07.294234 16238 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:07.286872 16232 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 12:12:07.297106 16238 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:07.298252 16123 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 12:12:07.298769 16185 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:07.370957 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.311718 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.315969 16172 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:07.325241 16170 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:07.329540 16116 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.214236 16074 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.208510 16040 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:07.344895 16034 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:07.346199 16211 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.212751 16067 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.159279 16193 raft_consensus.cc:2764] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 12:12:07.272395 16224 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.317s	user 0.002s	sys 0.000s
I20260709 12:12:07.366911 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:07.369037 11215 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:12:07.303885 16220 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 12:12:07.378680 16242 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.432787 16242 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.434269 16067 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:07.435573 16242 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.425741 16040 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:07.385143 15980 consensus_meta.cc:287] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.925s	user 0.005s	sys 0.008s
I20260709 12:12:07.439353 16040 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.385589 16221 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.386113 16243 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:07.377399 16172 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:07.391330 16200 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:07.399492 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 208853us
W20260709 12:12:07.453709 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 570021us
I20260709 12:12:07.461503 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.454450 16215 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:07.459532 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:07.372596 16123 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.470209 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.470523 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.473719 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.473869 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.474222 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.475147 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.478094 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.474874 10437 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
I20260709 12:12:07.474592 10436 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:07.479251 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.484560 10429 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:12:07.484431 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.485327 10438 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:07.485466 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.487118 10437 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:07.487267 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.489748 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.495266 11221 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:07.426023 16074 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.495029 16101 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:07.501117 16101 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.502147 16074 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:07.503515 16101 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:07.456897 16221 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.506214 16221 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:07.507375 16217 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:07.507527 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.508325 10578 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 8.
I20260709 12:12:07.509496 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.509735 16170 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:07.511122 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.508018 16217 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:07.512252 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50909.4us
I20260709 12:12:07.513767 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.514070 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.516023 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.516747 10436 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:12:07.510341 16170 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.515324 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:07.485146 16034 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:07.524142 10429 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:07.523087 10438 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
I20260709 12:12:07.526350 10429 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:07.531986 10434 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:12:07.533064 10437 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:12:07.525244 16221 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:12:07.541483 10429 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.547950 10429 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:07.548273 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.548681 10436 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:12:07.462339 16243 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.525887 16034 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:07.492260 16252 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:12:07.556448 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.549733 16215 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.528636 16045 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:07.550689 16221 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:07.564951 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.554891 16034 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:07.570735 16240 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:07.572345 16216 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.000s
I20260709 12:12:07.554289 16230 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:07.576687 16217 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:07.577118 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3087) took 1018 ms. Trace:
I20260709 12:12:07.579877 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.584504 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.585721 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.604434 10436 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:07.562793 16252 consensus_queue.cc:1243] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:12:07.576305 15995 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.007s
I20260709 12:12:07.590154 16160 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
W20260709 12:12:07.600123 16223 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.637s	user 0.004s	sys 0.000s
I20260709 12:12:07.590736 16172 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.607354 10434 raft_consensus.cc:2376] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11. Re-sending same reply.
I20260709 12:12:07.603317 10587 rpcz_store.cc:276] 0709 12:12:06.559002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.450857 (+891855us) service_pool.cc:224] Handling call
0709 12:12:07.451272 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:07.451717 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:07.451748 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:07.452289 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:12:07.524502 (+ 72213us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:07.524581 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.524681 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.576873 (+ 52192us) raft_consensus.cc:1685] finished
0709 12:12:07.576901 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.577051 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2674,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"thread_start_us":71754,"threads_started":1,"wal-append.queue_time_us":116340}
I20260709 12:12:07.609788 16240 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:07.584375 16221 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:12:07.590083 16117 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.619168 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.619904 10433 raft_consensus.cc:2376] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11. Re-sending same reply.
I20260709 12:12:07.613085 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:07.611447 10553 connection.cc:256] RPC call timeout handler was delayed by -1.5486693600000763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16570000000000001s for a response
I20260709 12:12:07.617071 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.617606 10438 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:07.622262 10438 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:07.622730 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:07.623469 10438 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.625070 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2964) took 1511 ms. Trace:
I20260709 12:12:07.627055 10438 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:07.627255 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.625522 10435 rpcz_store.cc:276] 0709 12:12:06.113790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.113933 (+   143us) service_pool.cc:224] Handling call
0709 12:12:06.362567 (+248634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.548992 (+186425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.549348 (+   356us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:06.619948 (+ 70600us) log.cc:844] Serialized 26 byte log entry
0709 12:12:06.728656 (+108708us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:06.728721 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.728811 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.599285 (+870474us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:07.624682 (+ 25397us) raft_consensus.cc:1685] finished
0709 12:12:07.624714 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.624965 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11227,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":32147200,"thread_start_us":220727,"threads_started":2,"wal-append.queue_time_us":301882}
W20260709 12:12:07.629215 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.629596 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:07.630959 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.631137 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:07.632990 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:07.634178 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:07.634565 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:07.635437 10505 raft_consensus.cc:2410] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:12:07.613135 16200 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.612499 15995 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:07.639496 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.639595 16055 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.641629 15995 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.642724 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.643661 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.643750 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.644163 10435 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 10.
I20260709 12:12:07.644747 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.645402 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.645601 10433 raft_consensus.cc:2410] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:07.646570 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:07.647122 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.648319 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.649700 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.650761 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.652194 16255 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.651643 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.652575 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.652786 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.653359 10434 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:07.654235 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:07.655757 10434 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:12:07.658025 10433 pending_rounds.cc:85] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 9
I20260709 12:12:07.658469 10433 pending_rounds.cc:107] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:12:07.658783 10433 raft_consensus.cc:2889] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:07.659662 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:07.662251 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.663734 10438 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:12:07.664151 10438 pending_rounds.cc:85] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 11
I20260709 12:12:07.664402 10438 pending_rounds.cc:107] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:12:07.664655 10438 raft_consensus.cc:2889] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:07.602766 16216 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:07.667797 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.669731 10403 leader_election.cc:400] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16.
I20260709 12:12:07.670138 10403 leader_election.cc:403] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:07.671571 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.676007 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.646066 16230 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:07.711347 16230 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:07.712297 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.701125 16191 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:07.716482 16225 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.755s	user 0.006s	sys 0.000s
I20260709 12:12:07.672633 16255 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:12:07.696487 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.703332 16193 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:12:07.704448 16228 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.742s	user 0.002s	sys 0.000s
I20260709 12:12:07.695322 16190 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:07.659948 16123 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.683240  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:07.707460 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3080) took 1184 ms. Trace:
I20260709 12:12:07.677538 16250 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 12:12:07.677966 16265 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:12:07.660178 16045 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:07.654506 16252 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:07.706568 10582 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2728) took 1770 ms (client timeout 1618 ms). Trace:
I20260709 12:12:07.672525 10429 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:12:07.719923 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:07.720956 10579 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:07.705171 16243 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.729521 10435 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:12:07.729965 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.672796 16215 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:07.731537 10584 rpcz_store.cc:276] 0709 12:12:06.523594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.524987 (+  1393us) service_pool.cc:224] Handling call
0709 12:12:06.539744 (+ 14757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:07.355255 (+815511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:07.355310 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.355379 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:07.355427 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.707120 (+351693us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b5400178c54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:07.707188 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.707375 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861916032}
I20260709 12:12:07.674820 16116 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16.
W20260709 12:12:07.735339 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.673087 16257 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:07.737926 11213 raft_consensus.cc:2410] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:12:07.727429 16217 raft_consensus.cc:2764] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:12:07.787499 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:07.738466 11217 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:07.737221 10429 pending_rounds.cc:85] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:12:07.791271 10429 pending_rounds.cc:107] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:12:07.791685 10429 raft_consensus.cc:2889] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:07.728137 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60433.6us
I20260709 12:12:07.740998 10432 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
W20260709 12:12:07.737052 10582 rpcz_store.cc:269] 0709 12:12:05.935982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.948075 (+ 12093us) service_pool.cc:224] Handling call
0709 12:12:07.706289 (+1758214us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540010c2d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:07.706448 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557227264}
I20260709 12:12:07.751077 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:07.797540 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46611.2us
I20260709 12:12:07.672559 16047 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:12:07.733553 16170 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:07.761420 16238 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:07.762749 10431 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 2985) took 1142 ms. Trace:
I20260709 12:12:07.756395 16215 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:07.728752 16193 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:07.804105 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.805323 10431 rpcz_store.cc:276] 0709 12:12:06.620001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.620158 (+   157us) service_pool.cc:224] Handling call
0709 12:12:06.620613 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.654507 (+ 33894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.654543 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:06.667510 (+ 12967us) log.cc:844] Serialized 26 byte log entry
0709 12:12:06.794430 (+126920us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:06.794489 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.794576 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.610284 (+815708us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:07.714092 (+103808us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b2400172c28. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa3109d 00007f2c9266310e 00007f2c9267a88c 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221
0709 12:12:07.762483 (+ 48391us) raft_consensus.cc:1685] finished
0709 12:12:07.762518 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.762677 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9645,"raft.queue_time_us":42011,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":23084,"spinlock_wait_cycles":81529984,"thread_start_us":70901,"threads_started":1,"wal-append.queue_time_us":269074}
I20260709 12:12:07.736335 16252 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:07.802374 16190 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:07.829437 16191 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:07.835270 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.843839 16193 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:07.812219 16215 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:07.905053 11178 leader_election.cc:336] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.168s (SENT)
I20260709 12:12:07.823137 16252 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:07.929687 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.902238 16242 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:07.934621 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 130611us
I20260709 12:12:07.910063 16243 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:07.941828 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:07.943120 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:07.958781 16115 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:07.958802 15913 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:07.961632 15913 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:07.964178 15913 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.964862 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 625) took 1899 ms. Trace:
I20260709 12:12:07.965356 10504 rpcz_store.cc:276] 0709 12:12:06.065702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.065837 (+   135us) service_pool.cc:224] Handling call
0709 12:12:06.429632 (+363795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.430414 (+   782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.964622 (+1534208us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54000daa50. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:07.964787 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835882880}
I20260709 12:12:07.942840 16242 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:07.967907 16216 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:07.968784 16230 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:07.969614 16242 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:07.970182 16242 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.936560 16193 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:07.937498 16187 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:08.026731 16034 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.038308 16034 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:08.039310 16272 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.064s	user 0.004s	sys 0.000s
W20260709 12:12:08.014770 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:08.046793 10553 leader_election.cc:336] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.077s (SENT)
I20260709 12:12:08.047453 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:07.930658 15699 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:12:08.016655 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74955.9us
I20260709 12:12:08.017423 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:07.970582 16208 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:08.031152 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:08.031929 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:08.033123 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:08.067196 10584 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:08.068668 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:08.033901 16273 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
W20260709 12:12:08.030226 16226 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.069s	user 0.006s	sys 0.000s
W20260709 12:12:08.072196 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.073326 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:08.088231 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:08.101321 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:08.079468 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:08.057703 16230 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.092177 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:08.115423 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:08.066313 15913 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:08.118026 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:08.120241 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:07.949054 16243 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.121984 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:08.116703 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:08.084519 16047 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:08.128337 16170 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:08.113049 16208 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:08.126740 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:08.149793 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3073) took 1680 ms. Trace:
I20260709 12:12:08.113464 16217 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:08.111109 16080 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:08.155272 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:08.156019 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:08.065196 15699 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.162688 10586 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.162650 10582 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:08.156406 10585 rpcz_store.cc:276] 0709 12:12:06.469283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.469416 (+   133us) service_pool.cc:224] Handling call
0709 12:12:06.483813 (+ 14397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:06.498000 (+ 14187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.498042 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:06.519518 (+ 21476us) log.cc:844] Serialized 52 byte log entry
0709 12:12:06.858174 (+338656us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:07.119188 (+261014us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.464677 (+345489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.464792 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.973716 (+508924us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:08.123602 (+149886us) raft_consensus.cc:1685] finished
0709 12:12:08.149404 (+ 25802us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.149695 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":390967,"prepare.queue_time_us":112686,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":4069,"raft.queue_time_us":381549,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":10124,"spinlock_wait_cycles":26353536,"thread_start_us":465483,"threads_started":2,"wal-append.queue_time_us":420341}
I20260709 12:12:08.165185 10585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:08.165820 10585 raft_consensus.cc:606] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:08.125203 16000 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:08.170934 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:08.171851 10510 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:08.168350 10585 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:08.117461 16221 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:08.181141 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 213895us
W20260709 12:12:08.189316 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 259782us
I20260709 12:12:08.126605 16230 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.193190 10510 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:08.168628 16170 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:08.173816 16210 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:08.207906 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:08.208069 16211 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:08.212507 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:08.213639 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:08.214514 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:08.215920 10512 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:08.240813 10510 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:08.242100 11215 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: []
I20260709 12:12:08.232089 16116 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:08.243325 16242 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:08.256100 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:08.256834 11213 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:08.170426 16080 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.241266 16121 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:08.262557 11214 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:12:08.241284 16210 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:08.267647 10586 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:08.267802 10582 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:08.294283 11213 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:08.249570 15883 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:12:08.258728 16242 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:12:08.302336 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 188315us
I20260709 12:12:08.250010 16116 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.259719 16123 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:12:08.379492 10582 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:08.382330 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3083) took 1843 ms. Trace:
I20260709 12:12:08.382872 10583 rpcz_store.cc:276] 0709 12:12:06.538780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.442956 (+904176us) service_pool.cc:224] Handling call
0709 12:12:07.443298 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:07.443627 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:07.478283 (+ 34656us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.094766 (+616483us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b180043ba00. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9d4e07 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f23a 00007f2c92ae9a2a
0709 12:12:08.381890 (+287124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:08.381962 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:08.382027 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.382116 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.382257 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35776,"prepare.queue_time_us":34616,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":2953,"spinlock_wait_cycles":16327936,"thread_start_us":286789,"threads_started":1}
I20260709 12:12:08.339954 16242 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:08.284653 16121 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:08.341403 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 261183us
I20260709 12:12:08.402591 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:08.385504 16216 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:08.431313 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:08.432420 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:08.432928 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:08.391880 16281 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.435668 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:08.439375 10438 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:12:08.445945 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3085) took 1894 ms. Trace:
I20260709 12:12:08.339756 16170 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.456782 11212 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:08.446496 10579 rpcz_store.cc:276] 0709 12:12:06.551524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.437688 (+886164us) service_pool.cc:224] Handling call
0709 12:12:07.438130 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:07.738361 (+300231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.445575 (+707214us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400090954. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:08.445848 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649765760}
I20260709 12:12:08.411314 16279 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:08.470570 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:08.475056 10430 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:08.421389 16243 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.419543 15913 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:08.443886 16252 consensus_meta.cc:287] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.001s
I20260709 12:12:08.513015 16252 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.368973 16210 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.523931 16210 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:08.389281 16280 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:12:08.469468 15883 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:12:08.463862 16281 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.492460 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:08.531844 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:08.493680 16257 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:08.494110 16045 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:08.494737 16230 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:08.485292 16279 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.500648 15913 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:08.506619 16121 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:08.508010 16116 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.511745 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:08.483269 16115 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.005s
I20260709 12:12:08.533398 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:08.552330 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 334430us
W20260709 12:12:08.555222 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 210473us
I20260709 12:12:08.559971 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:08.550705 16267 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:12:08.562968 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:08.571411 11212 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:08.558511 16291 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.572587 16240 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.601539 16291 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.602217 16240 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.603721 10578 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:08.570393 16230 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:08.555222 16045 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.583460 16115 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:08.571345 16159 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:12:08.571732 16217 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:12:08.632160 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 199815us
I20260709 12:12:08.613662 16230 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:08.633389 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:08.643998 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:08.630654 16252 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:08.635581 16080 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:08.637934 16286 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:12:08.638792 16230 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:08.660454 11221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 748) took 1480 ms. Trace:
I20260709 12:12:08.661008 11221 rpcz_store.cc:276] 0709 12:12:07.180501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.180985 (+   484us) service_pool.cc:224] Handling call
0709 12:12:07.181433 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:07.495793 (+314360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.660066 (+1164273us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400240ed4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:08.660351 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721116544}
I20260709 12:12:08.668536 16281 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.649189 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:08.670311 16215 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.755s	user 0.019s	sys 0.005s
I20260709 12:12:08.647229 16282 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:08.644791 16305 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:08.674929 16230 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.676288 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:08.682325 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:08.683223 16282 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:08.684079 16282 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:08.676491 16281 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:08.685392 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:08.649458 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:08.676893 16215 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.689847 10583 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:12:08.692114 16215 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:08.693506 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:08.695050 16295 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:12:08.696455 16230 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:08.696628 10585 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 2743) took 1031 ms. Trace:
I20260709 12:12:08.697000 10585 rpcz_store.cc:276] 0709 12:12:07.665187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.182331 (+517144us) service_pool.cc:224] Handling call
0709 12:12:08.183172 (+   841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:08.183494 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:08.183530 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:08.206618 (+ 23088us) log.cc:844] Serialized 26 byte log entry
0709 12:12:08.428276 (+221658us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:08.428344 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.428461 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.696365 (+267904us) raft_consensus.cc:1685] finished
0709 12:12:08.696391 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.696554 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":165306,"threads_started":1,"wal-append.queue_time_us":330478}
W20260709 12:12:08.706202 16193 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.734s	user 0.009s	sys 0.004s
I20260709 12:12:08.708127 16279 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:08.655938 16285 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 12:12:08.709103 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 230840us
I20260709 12:12:08.709828 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:08.710263 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:08.717311 10430 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:08.719798 10505 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:08.625471 16290 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:12:08.721711 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:08.722640 10512 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:08.724745 16279 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:08.725260 16279 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:08.725652 16279 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:08.726197 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:08.723839 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:08.736047 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:08.684516 16282 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:08.741173 10512 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:08.712134 16291 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:08.747329 16187 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:08.811614 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 243259us
I20260709 12:12:08.743113 16282 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:08.811245 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:08.821625 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 67784.7us
I20260709 12:12:08.752786 16303 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:08.808230 16294 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:12:08.753633 16281 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:08.832026 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:08.805315 16170 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:08.796205 16291 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:12:08.824674 16310 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:08.827399 16187 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.841976 16193 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:08.854447 16257 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.863587 16310 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:08.878016 16193 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:08.825882 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 87680.7us
I20260709 12:12:08.878679 16187 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.878207 16257 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.853118 16309 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:12:08.916616 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:08.919534 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:08.921974 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:08.922827 11219 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:08.924746 16115 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:08.929548 16291 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.930405 16291 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.933658 16291 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.935917 11216 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:08.937433 11221 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 14: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:08.851971 16281 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.884733 16242 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.882587 16170 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:08.850970 16303 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.970781 11220 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:08.951251 16305 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:08.952723 16045 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.878825 16310 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:08.961316 16193 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:08.988035 16187 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.992507 16305 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:08.997628 16115 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.047869 16315 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.048207 16313 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.048678 16315 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.048880 16313 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:09.051867 16319 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:12:09.056070 16288 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.056936 16242 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.056689 16288 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.013878 16187 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.059984 16159 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:09.060796 16315 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.060595 16159 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:09.062947 16159 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:09.075780 15699 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:09.054632 16282 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.078429 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54450 (request call id 634) took 1463 ms. Trace:
I20260709 12:12:09.078950 10436 rpcz_store.cc:276] 0709 12:12:07.614973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.620491 (+  5518us) service_pool.cc:224] Handling call
0709 12:12:09.077585 (+1457094us) spinlock_profiling.cc:243] Waited 970 us on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baeee 000000000036eb39 00007f2c97a2e1e0 00007f2c97af70c4 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:12:09.077792 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.078022 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:09.078051 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:09.078103 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:09.078134 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.078210 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:09.078356 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232960}
I20260709 12:12:09.084054 16170 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:09.084205 16305 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:09.087790 16325 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:12:09.087949 16305 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:09.089978 16282 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:09.090226 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 3102) took 1015 ms. Trace:
I20260709 12:12:09.090739 10578 rpcz_store.cc:276] 0709 12:12:08.075127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.126444 (+ 51317us) service_pool.cc:224] Handling call
0709 12:12:09.090021 (+963577us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54002774d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:09.090151 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023989504}
I20260709 12:12:09.091276 11220 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:09.100646 10504 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:09.105796 16159 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.106685 16159 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.109283 16115 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.111254 16214 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.136094 16214 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:09.138495 16214 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:09.148890 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:09.150122 10432 raft_consensus.cc:2368] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:12:09.158519 11178 leader_election.cc:400] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:12:09.159040 11178 leader_election.cc:403] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:12:09.140020 16313 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.134953 16159 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:09.167945 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 352045us
I20260709 12:12:09.173331 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.166347 16300 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:12:09.175446 16306 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:09.186009 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 216515us
I20260709 12:12:09.179692 16312 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.176601 15913 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:09.187594 16115 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.189219 16306 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:09.198841 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.208082 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:09.202768 16303 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:09.198679 15913 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.214308 16093 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:12:09.220582 16331 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:09.272876 15913 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:09.272989 16240 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.273540 16331 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:09.274004 16331 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:09.222236 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:12:09.275949 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53808.8us
I20260709 12:12:09.276465 16080 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:09.277707 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:09.278002 16257 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:09.222108 16314 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:09.235625 16306 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:09.243527 16292 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:09.278589 16138 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:09.279251 16292 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:09.279914 16160 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.280402 16093 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:09.280522 16160 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.280229 16333 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:09.281769 16333 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:09.282099 16333 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:09.284561 16160 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.285444 16138 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:09.285671 10583 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3099) took 1211 ms. Trace:
I20260709 12:12:09.285902 16138 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:09.286525 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.240998 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57890 (request call id 898) took 1525 ms. Trace:
I20260709 12:12:09.287326 10587 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:09.287408 11215 rpcz_store.cc:276] 0709 12:12:07.715489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.717770 (+  2281us) service_pool.cc:224] Handling call
0709 12:12:07.718086 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:08.242625 (+524539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.240655 (+998030us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540047a9d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:09.240897 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204439808}
I20260709 12:12:09.289525 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.274915 16331 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.291359 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:09.286846 16138 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:09.292176 10587 raft_consensus.cc:2368] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18.
I20260709 12:12:09.292634 16138 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:09.286093 10583 rpcz_store.cc:276] 0709 12:12:08.074268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.385555 (+311287us) service_pool.cc:224] Handling call
0709 12:12:08.404491 (+ 18936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:08.690244 (+285753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.285301 (+595057us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54003dc054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:09.285593 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655613696}
I20260709 12:12:09.293179 16216 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.293821 16216 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.294207 10581 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:09.251408 16221 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:09.258072 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 1.13398e+06us
I20260709 12:12:09.295674 16331 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:09.296514 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.258757 16200 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.297649 16200 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:09.298952 16279 consensus_meta.cc:287] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.009s
I20260709 12:12:09.299489 16279 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:09.300094 16200 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:09.301751 16279 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:09.301841 16216 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:09.302561 16216 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:09.302757 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:09.303552 10504 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:09.260087 16210 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.735s	user 0.016s	sys 0.009s
I20260709 12:12:09.213856 16301 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:09.214782 16310 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:09.307862 16310 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:09.308342 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.309262 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.238844 10478 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:09.310274 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71585.4us
I20260709 12:12:09.310796 10511 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:09.311877 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.316780 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.317524 16287 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:09.317983 16287 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:09.318349 16287 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:09.322322 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:09.323093 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.323637 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:09.324430 10513 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:09.308923 16310 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.280210 16292 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.327180 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.327803 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.329842 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.330049 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:09.332041 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.333895 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.335311 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:09.335688 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46075.4us
I20260709 12:12:09.336289 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.327343 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.330893 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.342778 10585 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:09.345991 10504 raft_consensus.cc:2410] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
W20260709 12:12:09.426295 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 88186.9us
I20260709 12:12:09.426608 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:09.426939 10512 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:09.428090 10505 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:09.435338 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.436385 10583 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:09.438576 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:09.440625 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.442232 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.443717 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.456827 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.457178 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.458138 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:09.462193 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.463943 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.464731 10585 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:12:09.463929 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.466529 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:09.478559 10513 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:09.480525 10512 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:12:09.487159 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.492857 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:09.495442 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.499724 10507 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:09.469326 16159 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:09.457036 16307 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.425513 15913 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
W20260709 12:12:09.512615 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.479313 16292 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:09.503902 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:09.496026 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:09.497215 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 341242us
I20260709 12:12:09.505278 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.505537 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:09.497582 10585 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:12:09.508421 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.425173 16255 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:12:09.469755 15976 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.511579 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.514359 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.534123 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.517558 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.488358 16309 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.523910 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.494587 16210 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.535873 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:09.509146 16190 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:09.494889 16250 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:09.539615 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:09.500756 16303 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.541996 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:09.542259 11214 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:09.503764 16116 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.546196 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:09.537109 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 261022us
I20260709 12:12:09.548727 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:09.552011 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.572409 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.542667 16117 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.550191 11212 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:12:09.563324 11214 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:12:09.565543 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.568050 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.554320 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.576535 10511 raft_consensus.cc:2368] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
I20260709 12:12:09.516359 16257 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.532707 16193 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.612008 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.544250 16159 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.622574 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.624281 10585 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:09.625670 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:09.548563 16160 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.630550 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.548106 16313 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.640936 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.642156 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.643322 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:09.632869 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:09.656086 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.516842 16307 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:09.622934 16305 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.534s	user 0.014s	sys 0.000s
I20260709 12:12:09.666296 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:09.667855 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 113651us
I20260709 12:12:09.675360 10437 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:09.558424 16210 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:09.658974 16304 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:12:09.585005 16214 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 12:12:09.550397 16309 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:09.586936 16303 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.647296 16055 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:09.608372 16116 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.588546 16250 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.698004 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:09.615084 16255 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:12:09.730587 10430 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:09.639603 16159 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.740183 10586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:09.707981 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:09.741578 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:09.708808 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 186918us
I20260709 12:12:09.664739 16299 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:09.614775 16117 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.712250 10429 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:12:09.722126 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 99135.5us
I20260709 12:12:09.755496 11177 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.615934 16257 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.741986 10581 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:12:09.749826 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:09.660671 16313 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:09.760087 10403 leader_election.cc:400] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18.
I20260709 12:12:09.763688 10403 leader_election.cc:403] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:12:09.767052 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:09.768435 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:09.781565 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:09.782330 10438 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:09.727037 16213 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.782889 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:09.783475 10430 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:09.786363 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:09.690418 15913 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:12:09.793886 10585 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:09.784849 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:09.838218 10583 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:09.839378 10586 raft_consensus.cc:606] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:09.830533 10430 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:09.839919 10586 raft_consensus.cc:614] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:09.847056 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:09.761335 16305 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:12:09.814551 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 159642us
W20260709 12:12:09.904237 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 148828us
I20260709 12:12:09.902578 10438 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:12:09.861918 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.781778 16255 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:09.803699 16341 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:12:09.838553 16336 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 12:12:09.957890 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:09.982952 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:09.960523 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 255397us
I20260709 12:12:09.801789 16193 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:09.946975 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:09.860059 16340 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:12:09.859227 16345 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:12:09.862762 16221 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:09.855216 16305 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:10.027493 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:12:10.033746 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 417413us
I20260709 12:12:10.004706 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:09.991864 16339 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.292s	user 0.000s	sys 0.002s
W20260709 12:12:10.034032 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 87148.3us
I20260709 12:12:09.837683 16213 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.037282 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:10.043566 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:10.044517 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.045536 11214 raft_consensus.cc:2410] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:12:09.777812 16190 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:09.992125 16355 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:10.003352 16119 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:10.003371 16193 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:09.993379 16282 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:10.069047 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:10.107978 11177 leader_election.cc:400] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16.
W20260709 12:12:10.082275 16336 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:12:10.033052 16343 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:10.109043 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:10.084522 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 43683.8us
I20260709 12:12:10.098803 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.128273 10430 raft_consensus.cc:2410] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:12:10.082561 16338 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:10.132861 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.090255 16221 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:10.027338 16353 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.125s
I20260709 12:12:10.084036 16357 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:10.086407 16351 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:12:10.145078 10587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.015s	sys 0.043s
W20260709 12:12:10.156774 10587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.015s	sys 0.044s
I20260709 12:12:10.093149 16355 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:10.114238 16193 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:10.115252 16242 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:10.113490 16282 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.092761 16190 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.179888 16349 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:10.140202 16221 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:10.154752 16210 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:10.222661 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 136280us
I20260709 12:12:10.161216 16303 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.173367 16119 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:10.166561 16351 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.196687 16307 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.208091 16282 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:10.220728 16190 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:10.238448 16221 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:10.143319 16305 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:10.237286 16349 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.264554 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:10.259301 16238 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:10.269759 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.246572 16250 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.248303 16309 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.252092 16159 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:10.271873 16333 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.989s	user 0.011s	sys 0.005s
I20260709 12:12:10.281842 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:10.284761 10433 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:10.296142 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.325066 10433 raft_consensus.cc:2410] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:12:10.283357 16349 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:10.336364 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 227395us
I20260709 12:12:10.341046 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:10.325031 16305 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:10.390512 16375 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:10.391721 16375 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:10.392179 16375 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.392632 16375 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:10.393996 16373 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:10.347935 16358 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.278s
I20260709 12:12:10.288189 16238 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.352523 16343 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:10.428442 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 228827us
I20260709 12:12:10.433075 16158 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:10.433588 16158 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:10.426525 16138 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.127s	user 0.019s	sys 0.004s
I20260709 12:12:10.264096 16355 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.322904 16333 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.424875 16303 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.422631 16357 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:10.455396 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:10.360731 16356 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.324s
I20260709 12:12:10.468039 10509 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:12:10.483731 10506 raft_consensus.cc:2368] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:12:10.434551 16158 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.456260 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3109) took 1140 ms. Trace:
I20260709 12:12:10.446800 16367 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.446043 16309 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:12:10.499843 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 206602us
I20260709 12:12:10.503233 11214 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:12:10.504591 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.461833 16351 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.487663 10587 rpcz_store.cc:276] 0709 12:12:09.316087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.324454 (+  8367us) service_pool.cc:224] Handling call
0709 12:12:09.331888 (+  7434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.332130 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:09.583798 (+251668us) log.cc:844] Serialized 14 byte log entry
0709 12:12:10.455855 (+872057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:10.455914 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:10.455975 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:10.456049 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:10.456190 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100256,"prepare.queue_time_us":164412,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":3899,"raft.queue_time_us":231510,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"spinlock_wait_cycles":153468160,"thread_start_us":977738,"threads_started":3,"wal-append.queue_time_us":691825}
I20260709 12:12:10.479658 16159 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.556699 16159 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:10.557824 16288 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:10.527616 16287 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.209s	user 0.010s	sys 0.006s
I20260709 12:12:10.559235 16287 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.550357 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:10.560503 16193 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.561182 10432 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:12:10.561640 16287 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:10.562431 16362 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.476s	user 0.004s	sys 0.001s
I20260709 12:12:10.454973 16303 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.565088 16117 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.565533 16303 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.566402 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 639) took 1062 ms. Trace:
I20260709 12:12:10.462461 16355 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:10.566721 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 635) took 1100 ms. Trace:
I20260709 12:12:10.545518 16257 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.562399 10512 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:10.567179 10509 rpcz_store.cc:276] 0709 12:12:09.466132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.466264 (+   132us) service_pool.cc:224] Handling call
0709 12:12:10.566544 (+1100280us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b540038efd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:10.566656 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2251905280}
I20260709 12:12:10.568429 10512 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 2692) took 1024 ms. Trace:
I20260709 12:12:10.568822 10512 rpcz_store.cc:276] 0709 12:12:09.543989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.546942 (+  2953us) service_pool.cc:224] Handling call
0709 12:12:10.568254 (+1021312us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54000da554. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:10.568365 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235842304}
I20260709 12:12:10.572455 16357 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.566759 10507 rpcz_store.cc:276] 0709 12:12:09.503576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.504267 (+   691us) service_pool.cc:224] Handling call
0709 12:12:09.504679 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:09.510134 (+  5455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:09.510176 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:09.522323 (+ 12147us) log.cc:844] Serialized 52 byte log entry
0709 12:12:09.795731 (+273408us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:09.795828 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.795919 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:10.558411 (+762492us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:10.566059 (+  7648us) raft_consensus.cc:1685] finished
0709 12:12:10.566081 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:10.566333 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6163,"raft.queue_time_us":5796,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":11347,"spinlock_wait_cycles":15256320,"thread_start_us":181141,"threads_started":1,"wal-append.queue_time_us":443410}
I20260709 12:12:10.575196 16213 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.520884 16158 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:10.527784 16354 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:12:10.491823 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 243558us
I20260709 12:12:10.572137 16193 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.595728 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.598809 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:10.599309 10403 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.000s	sys 0.012s
I20260709 12:12:10.601569 11214 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
W20260709 12:12:10.602183 10403 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.001s	sys 0.012s
W20260709 12:12:10.602955 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 818143us
I20260709 12:12:10.603449 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:10.607301 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.608014 10507 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:10.571914 16238 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:10.588169 16292 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 1.046s	user 0.013s	sys 0.002s
I20260709 12:12:10.588166 16303 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.603917 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:10.612435 16045 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:10.612767 16250 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:10.526333 16309 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.595376 16257 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.614156 16351 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:10.632185 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:10.634685 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:10.635522 10435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.006s	sys 0.009s
W20260709 12:12:10.636047 10435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.006s	sys 0.009s
I20260709 12:12:10.641501 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:10.638260 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.621738 16045 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.519707 16367 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:10.647898 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:10.634931 16351 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:10.631917 16193 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:10.635916 16310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.011s	sys 0.011s
I20260709 12:12:10.659698 10578 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:10.661235 16350 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:10.624707 16303 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.662195 16377 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.664511 16377 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.663420 16238 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.665683 16238 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.666851 16281 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.636705 16257 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.649134 16045 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:10.693239 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.694221 11213 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:10.694926 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:10.695709 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:10.660980 16310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.011s	sys 0.011s
I20260709 12:12:10.657078 16351 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:10.660692 16331 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.364s	user 0.008s	sys 0.009s
I20260709 12:12:10.699735 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:10.704635 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.704685 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:10.699959 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:10.676903 16381 raft_consensus.cc:993] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:12:10.718731 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.723892 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:10.726420 10504 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:12:10.677887 16350 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:10.670017 16034 raft_consensus.cc:3055] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:12:10.671346 16387 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.730942 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:10.675925 16193 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.737666 11213 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: []
I20260709 12:12:10.718259 16281 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:10.632596 16250 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.666968 16377 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.681195 16257 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:10.750337 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76069.3us
I20260709 12:12:10.668346 16238 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.675742 16349 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.713017 15838 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:12:10.735424 16000 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:10.743973 16333 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.744740 16193 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.766945 10432 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:10.767266 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:10.767968 11214 raft_consensus.cc:2376] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16. Re-sending same reply.
I20260709 12:12:10.730630 16387 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:10.744576 16255 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.005s
I20260709 12:12:10.761195 16383 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:10.734097 16355 raft_consensus.cc:1081] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:12:10.732254 16034 raft_consensus.cc:740] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:10.740461 16386 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:10.682686 16221 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.743498 16210 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:10.731271 16190 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.777470 16255 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18.
I20260709 12:12:10.749888 16288 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:12:10.781602 16386 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:10.794426 11220 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:10.780643 16383 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:10.780320 16353 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.804785 10587 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:10.790813 16187 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.799141 16386 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:10.813036 16116 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.814079 15699 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:10.816778 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2399) took 3170 ms. Trace:
I20260709 12:12:10.802354 16383 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:10.819629 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2759) took 1490 ms. Trace:
I20260709 12:12:10.817363 16190 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.817175 10434 rpcz_store.cc:276] 0709 12:12:07.645947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.671423 (+ 25476us) service_pool.cc:224] Handling call
0709 12:12:07.799743 (+128320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:07.800365 (+   622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:07.800429 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:07.802996 (+  2567us) log.cc:844] Serialized 26 byte log entry
0709 12:12:08.076944 (+273948us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:08.077237 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.346875 (+269638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.348283 (+  1408us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:09.306294 (+958011us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b1800332320. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:09.313640 (+  7346us) raft_consensus.cc:1685] finished
0709 12:12:09.313752 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:10.816552 (+1502800us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400001750. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:10.816677 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53007,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":113772,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":42862,"spinlock_wait_cycles":308345472,"thread_start_us":516754,"threads_started":2,"wal-append.queue_time_us":357756,"wal-append.run_cpu_time_us":2743,"wal-append.run_wall_time_us":1018227}
I20260709 12:12:10.821280 10587 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:12:10.821280 10504 rpcz_store.cc:276] 0709 12:12:09.329114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.347842 (+ 18728us) service_pool.cc:224] Handling call
0709 12:12:09.348323 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:10.726838 (+1378515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:10.819310 (+ 92472us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002c6ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:10.819557 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3168652544}
I20260709 12:12:10.789001 16350 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.826263 16231 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:10.831779 16231 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.834256 16231 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.835533 16387 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.837508 16313 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.831202 16301 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.839471 16301 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.841751 16301 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.831200 16303 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.843457 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.844473 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.844923 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.856788 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:10.857561 10587 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:10.859028 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:10.855080 16368 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.806152 16353 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:10.881757 16342 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 1.133s	user 0.003s	sys 0.000s
I20260709 12:12:10.883311 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3104) took 2807 ms. Trace:
I20260709 12:12:10.883512 16309 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.884133 16221 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.863118 16310 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:10.885617 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.883734 10579 rpcz_store.cc:276] 0709 12:12:08.075838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.467809 (+391971us) service_pool.cc:224] Handling call
0709 12:12:08.489293 (+ 21484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:08.489816 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:08.489860 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:09.241293 (+751433us) spinlock_profiling.cc:243] Waited 964 us on lock 0x7b2c000dfe08. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:12:09.241956 (+   663us) log.cc:844] Serialized 52 byte log entry
0709 12:12:09.547700 (+305744us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:09.547756 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.547858 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:10.881099 (+1333241us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:10.882946 (+  1847us) raft_consensus.cc:1685] finished
0709 12:12:10.882977 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:10.883206 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25369,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":125,"spinlock_wait_cycles":17882752,"thread_start_us":295765,"threads_started":1,"wal-append.queue_time_us":434389}
I20260709 12:12:10.896697 16353 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.899492 10436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 3003) took 1178 ms. Trace:
W20260709 12:12:10.900475 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:10.899948 10436 rpcz_store.cc:276] 0709 12:12:09.721441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.721644 (+   203us) service_pool.cc:224] Handling call
0709 12:12:10.681564 (+959920us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bae9c 00007f2c8f9baa5b 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:12:10.681768 (+   204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:10.682147 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:10.682195 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:10.682782 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:12:10.825309 (+142527us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:10.825373 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:10.825471 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:10.899207 (+ 73736us) raft_consensus.cc:1685] finished
0709 12:12:10.899229 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:10.899425 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10657,"raft.queue_time_us":1485,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":12013,"spinlock_wait_cycles":29162624,"thread_start_us":142046,"threads_started":1,"wal-append.queue_time_us":210151}
I20260709 12:12:10.890311 16310 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.911500 16310 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:10.900075 16282 consensus_meta.cc:287] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.003s
W20260709 12:12:10.915939 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:10.919076 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.921530 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.924065 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.886152 16368 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.926875 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:10.915701 16282 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.929556 10431 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:10.932794 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.933808 10504 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:10.930276 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.927347 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.927675 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.927886 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:10.928526 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.940198 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.941414 10435 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:10.927547 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.935060 16275 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.936160 16309 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:10.943385 16275 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:10.943511 16309 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.946647 10507 raft_consensus.cc:2410] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:10.949369 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:10.949424 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.951382 10437 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:10.953282 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:10.954044 10434 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:10.873001 16351 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.955150 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:10.955356 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.955631 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:10.956157 10435 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 9.
I20260709 12:12:10.955029 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:10.954264 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.957139 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.963305 10430 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:10.931727 16191 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:10.954838 16305 consensus_meta.cc:287] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.004s
I20260709 12:12:10.969051 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:10.977314 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:10.945900 16309 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.984032 10431 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:12:10.984934 10578 raft_consensus.cc:2368] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:12:10.989013 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:10.989770 10509 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:10.956096 16353 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:12:10.956449 16303 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.996059 16303 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:10.998466 16303 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.963157 16190 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:10.965905 16055 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.968706 16367 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.006136 10509 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:10.961656 16355 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.976756 16368 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:10.976004 16275 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:10.971800 16377 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.001557 16351 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:10.958884 16045 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.015321 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.015106 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.015298 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:11.002542 16387 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:10.975613 16191 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.025359 10434 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:12:11.027460 10433 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:11.013476 16356 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.029210 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.033082 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 3010) took 1271 ms. Trace:
I20260709 12:12:11.033059 16310 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.036504 16191 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:11.023981 16375 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.006s
I20260709 12:12:11.040136 10581 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3126) took 1541 ms. Trace:
I20260709 12:12:11.060783 11212 raft_consensus.cc:1240] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 8. Current term is 9. Ops: []
I20260709 12:12:11.038828 10429 rpcz_store.cc:276] 0709 12:12:09.761944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.762245 (+   301us) service_pool.cc:224] Handling call
0709 12:12:11.032217 (+1269972us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b5800000c98. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9baa22 00007f2c988d38d9 00007f2c97a2dee0 00007f2c97af70ab 00007f2c97af7131 00007f2c97ab8fc0 00007f2c97aa3929 00007f2c97aa33d2 00007f2c9101abd4
0709 12:12:11.032496 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:11.032719 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:11.032750 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:11.032801 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:11.032839 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.032897 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.033012 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13204736}
I20260709 12:12:11.061223 10581 rpcz_store.cc:276] 0709 12:12:09.498673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.758049 (+259376us) service_pool.cc:224] Handling call
0709 12:12:09.758303 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.839126 (+ 80823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:09.839165 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:09.839213 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:09.839247 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.036040 (+1196793us) spinlock_profiling.cc:243] Waited 80.5 ms on lock 0x7b54001caf54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:11.039887 (+  3847us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.040050 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185262848}
I20260709 12:12:11.044770 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.062960 10433 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:11.046581 16282 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.063292 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.022405 16377 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.026921 16387 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.025801 16357 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:11.057097 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 3020) took 1098 ms. Trace:
I20260709 12:12:11.121269 10430 rpcz_store.cc:276] 0709 12:12:09.958962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.151974 (+193012us) service_pool.cc:224] Handling call
0709 12:12:10.152467 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:10.965921 (+813454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.056749 (+ 90828us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b54002652d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:11.057013 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854343936}
W20260709 12:12:11.122310 16369 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.877s	user 0.004s	sys 0.000s
I20260709 12:12:11.038964 10436 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:11.017022 16355 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.064289 16356 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.013522 16055 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.125720 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:11.129150 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.130038 10433 raft_consensus.cc:2393] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:11.065820 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.012248 16190 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:11.124675 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:11.064854 16400 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:12:11.131816 10430 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:11.135535 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:11.136286 10433 raft_consensus.cc:2376] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16. Re-sending same reply.
I20260709 12:12:11.136626 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.133221 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.137521 10587 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:11.137324 11212 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:11.142107 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.142602 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:11.143114 10433 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:11.143486 10429 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:11.144677 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.144930 10436 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:11.145301 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:11.146135 10429 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:12:11.147106 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.147938 10433 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:11.151115 10587 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:11.161527 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.164639 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.165686 10581 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:12:11.166929 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:11.167781 10431 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:11.168479 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:11.169096 11214 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:11.170881 16355 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.064284 16375 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.173094 16356 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.173573 16375 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.174038 10580 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:11.175565 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3117) took 1848 ms. Trace:
I20260709 12:12:11.175670 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.175948 10580 rpcz_store.cc:276] 0709 12:12:09.326807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.722958 (+1396151us) service_pool.cc:224] Handling call
0709 12:12:10.723379 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:11.174519 (+451140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.175303 (+   784us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54001d56d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:11.175502 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1035924864}
I20260709 12:12:11.176533 16190 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.177287 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.158933 16093 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:11.179939 16093 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.180713 16310 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:11.178350 16401 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:12:11.132162 16279 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:11.185319 11214 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:11.188840 16045 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.190186 16193 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.191761 16055 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.192641 10506 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 633) took 1860 ms. Trace:
I20260709 12:12:11.193186 10506 rpcz_store.cc:276] 0709 12:12:09.332392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.336028 (+  3636us) service_pool.cc:224] Handling call
0709 12:12:11.192248 (+1856220us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540039f8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:11.192546 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2631359744}
I20260709 12:12:11.195657 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 3026) took 1037 ms. Trace:
I20260709 12:12:11.196162 10432 rpcz_store.cc:276] 0709 12:12:10.159240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.569988 (+410748us) service_pool.cc:224] Handling call
0709 12:12:10.570301 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:10.767465 (+197164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.195289 (+427824us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400438354. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:11.195567 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451752192}
I20260709 12:12:11.203734 10510 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:11.205065 10580 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:11.131320 16115 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:11.344218 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:11.348134 10430 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:11.165815 10433 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:11.353960 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 197241us
I20260709 12:12:11.220670 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.232672 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:11.348183 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:11.413926 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.252027 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.213174 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:11.265048 10583 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2755) took 1750 ms (client timeout 1581 ms). Trace:
W20260709 12:12:11.232564 10585 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 3128) took 1722 ms (client timeout 2187 ms). Trace:
I20260709 12:12:11.262817 10586 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49158 (request call id 20) took 1589 ms. Trace:
I20260709 12:12:11.282122 11217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57904 (request call id 760) took 1238 ms. Trace:
I20260709 12:12:11.424002 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.425384 16193 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.373294 10584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3112) took 2054 ms. Trace:
I20260709 12:12:11.381292 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:11.227881 16093 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.127718 16282 consensus_queue.cc:1059] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:12:11.419387 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.231406 16250 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.434408 10578 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:11.413841 16299 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:11.420954 10586 rpcz_store.cc:276] 0709 12:12:09.673551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.712241 (+ 38690us) service_pool.cc:224] Handling call
0709 12:12:11.262528 (+1550287us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54001caf54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ad9853 00007f2c97aa760a 00007f2c9102d7d4 00007f2c9102d762 00007f2c9102d691 00007f2c9102d60c 00007f2c9102c8c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:11.262706 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226281728}
I20260709 12:12:11.436630 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.436723 16299 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.438228 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.438400 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:11.252977 16344 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.904s	user 0.001s	sys 0.002s
I20260709 12:12:11.301970 16349 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.142521 16357 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:11.250193 16361 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.910s	user 0.000s	sys 0.003s
W20260709 12:12:11.278280 16370 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.914s	user 0.002s	sys 0.001s
I20260709 12:12:11.196825 16377 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.450131 10438 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 3019) took 1496 ms. Trace:
I20260709 12:12:11.450618 10438 rpcz_store.cc:276] 0709 12:12:09.954303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.954474 (+   171us) service_pool.cc:224] Handling call
0709 12:12:09.954851 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:10.028621 (+ 73770us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:10.028745 (+   124us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:10.030131 (+  1386us) log.cc:844] Serialized 26 byte log entry
0709 12:12:10.164417 (+134286us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:10.164466 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:10.164550 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:11.277549 (+1112999us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:11.449467 (+171918us) raft_consensus.cc:1685] finished
0709 12:12:11.449509 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.449886 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":287377,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":14395,"spinlock_wait_cycles":106619136,"thread_start_us":206796,"threads_started":2,"wal-append.queue_time_us":265815}
I20260709 12:12:11.423898 11217 rpcz_store.cc:276] 0709 12:12:10.043505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.043688 (+   183us) service_pool.cc:224] Handling call
0709 12:12:10.086518 (+ 42830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:10.160934 (+ 74416us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:12:10.614543 (+453609us) log.cc:844] Serialized 14 byte log entry
0709 12:12:10.752969 (+138426us) log.cc:844] Serialized 14 byte log entry
0709 12:12:10.753201 (+   232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:10.753256 (+    55us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:12:10.753308 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:10.876501 (+123193us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b180036ec20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:10.877617 (+  1116us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b180036ec20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:11.266845 (+389228us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b180036ec20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419
0709 12:12:11.281886 (+ 15041us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b5400591854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:11.281927 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.282060 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":511413,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":108,"raft.queue_time_us":2796,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":9250,"spinlock_wait_cycles":243355776,"thread_start_us":516861,"threads_started":2,"wal-append.queue_time_us":148715}
I20260709 12:12:11.261963 16383 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.426529 16193 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:11.456084 16383 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.457100 16383 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.458000 16383 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.292590 16387 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.276805 16386 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.386085 16313 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.473966 16313 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.475150 16357 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.477290 16313 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:11.236850 16400 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:12:11.479176 10435 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:11.479637 10581 raft_consensus.cc:2393] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 20: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:12:11.236285 16404 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:12:11.480734 16282 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.481633 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2693) took 1914 ms. Trace:
W20260709 12:12:11.229703 16406 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.078s	user 0.004s	sys 0.000s
I20260709 12:12:11.481966 16282 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.383949 16045 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.482209 10510 rpcz_store.cc:276] 0709 12:12:09.567088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.572179 (+  5091us) service_pool.cc:224] Handling call
0709 12:12:11.203419 (+1631240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:11.211518 (+  8099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.481031 (+269513us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540039f8d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:11.481326 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3750514176}
I20260709 12:12:11.483566 16045 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.427539 10584 rpcz_store.cc:276] 0709 12:12:09.318444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.320205 (+  1761us) service_pool.cc:224] Handling call
0709 12:12:09.320565 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.497918 (+177353us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:09.511347 (+ 13429us) log.cc:844] Serialized 14 byte log entry
0709 12:12:09.722709 (+211362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:09.722789 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:09.722861 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.372436 (+1649575us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54001a1154. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:11.372543 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.373005 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17059,"prepare.queue_time_us":90916,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":5799,"raft.queue_time_us":233472,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":30875,"spinlock_wait_cycles":407220480,"thread_start_us":210932,"threads_started":1,"wal-append.queue_time_us":600046,"wal-append.run_cpu_time_us":1706,"wal-append.run_wall_time_us":1054246}
I20260709 12:12:11.485526 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:11.486495 10584 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.297858 16159 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.400792 16187 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.414690 16291 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.414847 16160 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.417359 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.417832 10579 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:12:11.425374 10508 raft_consensus.cc:1813] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Handling vote request from an unknown peer 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:11.418835 10580 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:11.420488 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:11.419314 10583 rpcz_store.cc:269] 0709 12:12:09.515138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.553733 (+ 38595us) service_pool.cc:224] Handling call
0709 12:12:11.264812 (+1711079us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b540010c7d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:11.264957 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516465920}
W20260709 12:12:11.420077 10585 rpcz_store.cc:269] 0709 12:12:09.510322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.640662 (+130340us) service_pool.cc:224] Handling call
0709 12:12:11.232363 (+1591701us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400179654. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:11.232495 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337345920}
I20260709 12:12:11.427069 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:11.465363 10578 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:12:11.493338 10508 raft_consensus.cc:2410] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:12:11.493364 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.498706 10506 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 18: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:11.502568 16160 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.504230 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.505944 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.506429 16160 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.507905 16187 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.509624 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.512293 11215 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 734) took 1009 ms. Trace:
I20260709 12:12:11.512152 16291 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.495855 10586 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:11.490842 16045 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.514063 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:11.483434 16386 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.524014 10505 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:11.513332 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.521651 11215 rpcz_store.cc:276] 0709 12:12:10.503133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.503281 (+   148us) service_pool.cc:224] Handling call
0709 12:12:10.503847 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:10.504309 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:10.504337 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:10.921735 (+417398us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4c001b6440. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c90daba20 00007f2c90dbca29 00007f2c90d82644 00007f2c90da9f2d 00007f2c90da9b93 00007f2c92ada18b 00007f2c92adacb5 00007f2c92adc004 00007f2c92adb729 00007f2c92ae0d96 00007f2c92ad5adb
0709 12:12:10.922607 (+   872us) log.cc:844] Serialized 548 byte log entry
0709 12:12:11.007431 (+ 84824us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:11.007474 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.007547 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:11.186501 (+178954us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:11.511629 (+325128us) raft_consensus.cc:1685] finished
0709 12:12:11.511697 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.512003 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44168704,"thread_start_us":84513,"threads_started":1,"wal-append.queue_time_us":120457}
I20260709 12:12:11.536078 10585 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:11.530246 16282 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.543408 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:11.557166 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.558770 10513 raft_consensus.cc:2368] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:12:11.561510 16034 consensus_queue.cc:260] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.563252 16187 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.564780 16299 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.568444 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.574460 16034 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:12:11.575924 11215 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:12:11.587534 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.600934 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:11.602046 10510 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.607745 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.618062 10580 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
W20260709 12:12:11.623680 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:11.628876 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.631088 10513 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:12:11.629940 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.636273 16291 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.625114 11212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57892
I20260709 12:12:11.641150 11212 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:11.641849 11212 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.642292 11215 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:12:11.648648 16416 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.081s	user 0.004s	sys 0.000s
I20260709 12:12:11.649544 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55480 (request call id 3127) took 2147 ms. Trace:
I20260709 12:12:11.627159 16115 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.650532 10583 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:11.649988 10582 rpcz_store.cc:276] 0709 12:12:09.501555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.774282 (+272727us) service_pool.cc:224] Handling call
0709 12:12:11.265462 (+1491180us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:11.265931 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:11.265965 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:11.266679 (+   714us) log.cc:844] Serialized 52 byte log entry
0709 12:12:11.409316 (+142637us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:11.547179 (+137863us) log.cc:844] Serialized 14 byte log entry
0709 12:12:11.555646 (+  8467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.574389 (+ 18743us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b1800457960. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:11.590613 (+ 16224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:11.590671 (+    58us) raft_consensus.cc:1685] finished
0709 12:12:11.592991 (+  2320us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.649121 (+ 56130us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540010c7d0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ade8ad 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:11.649277 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27481,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":9488,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":360,"spinlock_wait_cycles":3457687552,"thread_start_us":127099,"threads_started":1,"wal-append.queue_time_us":180245}
I20260709 12:12:11.663014 16279 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.662039 10331 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:11.663883 16279 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.682838 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.683055 16187 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:11.682919 16305 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.685947 16305 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.687438 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:11.688783 16305 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.691607 16359 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:11.659487 16193 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.694720 15995 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.695541 15995 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.698436 15995 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.683871 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:11.705188 16115 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.712181 16299 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:11.713344 10579 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:11.714761 16299 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:11.715665 16299 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:11.650743 16383 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.716981 16299 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.719075 16299 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.721683 16305 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:11.722271 16305 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:11.723448 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:11.730544 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47050.5us
I20260709 12:12:11.733430 11217 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:11.734688 16386 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:11.735255 16187 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.736232 16187 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.737350 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.737877 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.739856 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:11.740476 11217 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:11.746533 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.747845 10583 raft_consensus.cc:2393] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 22: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:11.750525 16187 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.757984 15995 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:11.759346 16291 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:11.759791 16291 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:11.761169 16291 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.762873 16291 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:11.768842 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.813565 16383 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:11.814582 16383 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.816416 16115 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:11.817373 16115 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.827039 16115 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.827458 10510 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:12:11.829903 16383 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:11.670662 10331 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 13, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:11.840420 11217 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
W20260709 12:12:11.841768 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62471.9us
I20260709 12:12:11.849668 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.744457 16279 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:11.870882 10584 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:11.877460 10581 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:11.845679 16305 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.880980 10505 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:11.888000 16093 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:11.889436 15995 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:11.889916 15995 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:11.894193 11212 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:11.897271 11215 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:11.911635 16305 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:11.913113 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48890.8us
I20260709 12:12:11.915071 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.915866 16373 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:11.920231 16221 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:11.921875 16221 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:11.922348 16221 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:11.922782 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.923527 10583 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:11.924582 10478 leader_election.cc:400] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:12:11.924949 10478 leader_election.cc:403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:12:11.924990 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.926059 11217 raft_consensus.cc:2393] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 9: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:11.928010 16093 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:12:11.931607 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:11.933794 16417 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
I20260709 12:12:11.934486 11212 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:11.935624 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:11.935742 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:11.935734 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:11.937975 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.941075 10581 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:11.941505 10581 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:11.942680 10581 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:11.943331 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:11.944717 10581 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.944607 15995 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.946292 15995 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:11.950830 10583 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:11.955575 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:11.962148 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.961738 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.964825 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.966704 16221 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:11.967198 11177 leader_election.cc:336] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.615s (SENT)
I20260709 12:12:11.968261 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:11.957422 10331 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 12, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 12 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:11.968259 16221 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.972630 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:11.978451 11215 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:11.979816 16313 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.981559 16313 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:11.984706 16313 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:11.987164 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:11.989512 10513 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:12:11.968637 16359 consensus_queue.cc:1175] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 16 status { last_received { term: 15 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:11.987387 10331 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 13, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:12.007050 10331 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 11, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:12.010948 10331 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 14 to 13, leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:12:12.011921 10336 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 225) took 19495 ms (client timeout 15000 ms). Trace:
W20260709 12:12:12.015214 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.020314 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:12.028347 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104288us
I20260709 12:12:12.035609 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:12.036664 16313 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:12.038305 16313 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:12.038769 16313 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:12.040189 16313 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:12.043821 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76873.6us
I20260709 12:12:12.044085 16313 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:12.049440 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 131259us
W20260709 12:12:12.050400 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.552s (SENT) [suppressed 17 similar messages]
I20260709 12:12:12.051390 16193 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.051653 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:12.052356 10510 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:12.052083 16193 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.025699 16359 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:12.004820 16093 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:12:12.014905 10336 rpcz_store.cc:269] 0709 12:11:52.516442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.517409 (+   967us) service_pool.cc:224] Handling call
0709 12:11:52.668923 (+151514us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b5400270010. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb0b2 00007f2c98934759 00007f2c988cca76 00007f2c9889dd58 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc
0709 12:12:12.011602 (+19342679us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:03.510259 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:03.511102 (+   843us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:03.511162 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:12:03.511181 (+    19us) tablet.cc:662] Decoding operations
0709 12:12:03.513303 (+  2122us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:03.513414 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:03.513447 (+    33us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:12:03.695602 (+182155us) tablet.cc:701] Row locks acquired
0709 12:12:03.695662 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:12:03.695908 (+   246us) write_op.cc:270] Start()
0709 12:12:03.696163 (+   255us) write_op.cc:276] Timestamp: P: 1783599123695815 usec, L: 0
0709 12:12:03.696215 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 12:12:03.697188 (+   973us) log.cc:844] Serialized 9701 byte log entry
0709 12:12:04.735443 (+1038255us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:05.321724 (+586281us) op_driver.cc:464] REPLICATION: finished
0709 12:12:05.569719 (+247995us) write_op.cc:301] APPLY: starting
0709 12:12:05.569876 (+   157us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:06.045420 (+475544us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adc32a 00007f2c92a61f14 00007f2c92a61ec3 00007f2c92a61e38 00007f2c92a61df3 00007f2c92a610dc 00007f2c92a8ae16 00007f2c92a72cbb 00007f2c92a5d39d 00007f2c92a60d75
0709 12:12:06.414885 (+369465us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:06.414950 (+    65us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:08.042511 (+1627561us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:08.042968 (+   457us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 12:12:08.043051 (+    83us) write_op.cc:312] APPLY: finished
0709 12:12:08.043731 (+   680us) log.cc:844] Serialized 220 byte log entry
0709 12:12:08.412903 (+369172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:08.435132 (+ 22229us) write_op.cc:454] Released schema lock
0709 12:12:08.475127 (+ 39995us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:08.511460 (+ 36333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:08.511512 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":830390,"reported_tablets":52,"spinlock_wait_cycles":7244928,"child_traces":[["op",{"apply.queue_time_us":230134,"apply.run_cpu_time_us":103708,"apply.run_wall_time_us":2941712,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":78217,"prepare.run_cpu_time_us":19586,"prepare.run_wall_time_us":436568,"raft.queue_time_us":447798,"raft.run_cpu_time_us":23757,"raft.run_wall_time_us":765217,"replication_time_us":1612171,"spinlock_wait_cycles":72989824,"thread_start_us":1216976,"threads_started":5,"wal-append.queue_time_us":742932,"wal-append.run_cpu_time_us":43867,"wal-append.run_wall_time_us":3231431}]]}
I20260709 12:12:12.061086 10513 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:12:12.060856 10331 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 14, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 14 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:12.069459 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:12.091890 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:12.093133 10583 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
W20260709 12:12:12.094002 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43713.4us
I20260709 12:12:12.079074 10331 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:12.095255 16425 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:12.099712 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64410.2us
I20260709 12:12:12.101286 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:12.105463 16425 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:12.107366 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:12.113073 10479 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:12.113530 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:12.116318 10581 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
W20260709 12:12:12.119684 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68784us
I20260709 12:12:12.119494 10505 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:12.126755 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.127926 16383 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:12.128386 16383 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.128796 16383 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:12.129252 15847 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:12.132550 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.119529 10331 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:12.137674 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:12.148355 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.149576 16115 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.150323 16115 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.151484 16373 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 7 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:12.152925 16115 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.154668 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:12.155542 10581 raft_consensus.cc:2368] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 19. Current term is 20.
I20260709 12:12:12.155951 16383 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.160563 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:12.163465 10584 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:12.178941 10507 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:12.181454 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:12.185702 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:12.185905 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:12.186605 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:12.187584 10513 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:12:12.189168 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:12.190742 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.178179 16305 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:12.184078 16115 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:12.196753 10510 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:12.201314 10509 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:12.193374 16305 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.202559 16305 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:12.211215 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 97956.3us
I20260709 12:12:12.217861 16353 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:12.218590 16353 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:12.219877 16353 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.206238 10331 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 16, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:12.227451 10505 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:12.208300 16115 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:12.251295 16193 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.252830 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:12.253355 16383 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:12.253692 11215 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 9.
I20260709 12:12:12.257112 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.258709 16193 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:12:12.259202 16193 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.231508 16428 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:12.272148 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59135.9us
I20260709 12:12:12.272835 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:12.273931 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.279237 16383 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:12.279776 16383 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.280156 16383 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:12.283186 16435 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:12.283463 16187 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:12.236737 16313 consensus_queue.cc:1059] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:12:12.287568 16313 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:12.287861 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:12.288120 16313 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:12.289674 16313 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.291766 15847 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:12.292940 15847 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:12.294445 15847 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:12.295352 16431 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:12.295674 16093 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:12.296118 16431 consensus_queue.cc:1243] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:12:12.296223 16093 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:12.297521 16093 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.283219 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.299837 16305 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.299966 16115 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.300866 16187 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:12:12.301327 16187 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.301743 16187 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:12.302250 16305 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:12.300209 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.304550 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:12.294902 15847 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:12.305383 16034 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:12.305847 16034 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.306208 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:12.306915 10505 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:12.311247 10552 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.312181 16435 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:12.312585 16435 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.312951 16435 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:12.307482 15847 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.318789 16428 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.274245 10553 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:12.323076 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 97648.9us
I20260709 12:12:12.265827 16193 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:12:12.326491 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:12.355675 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.360059 10509 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:12:12.361430 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45114.9us
I20260709 12:12:12.306248 16034 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:12.321465 16313 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:12.374593 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:12.379411 16240 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:12.380146 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.382648 16430 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:12.383875 16430 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:12.384579 16422 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:12.384768 16431 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:12.385232 16422 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:12.385277 16431 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:12.388425 16431 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.389976 16431 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:12.393746 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 71294.5us
I20260709 12:12:12.364290 16305 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:12.407788 10505 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:12:12.380163 16240 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:12.416996 10552 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.423110 16187 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.423764 16240 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.425717 16187 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:12.426432 16187 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:12.415133 16093 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:12.414515 16383 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:12.431195 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 56658us
I20260709 12:12:12.437930 16428 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:12.435231 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:12.409911 16305 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:12.446103 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:12.448493 10505 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:12.446255 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66259.6us
W20260709 12:12:12.471246 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.426815 16187 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:12.473834 16432 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:12:12.386543 16422 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:12.478155 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 211418us
I20260709 12:12:12.445241 10513 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 7 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:12.470268 16305 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:12.489260 10478 leader_election.cc:400] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 19. Current term is 20.
I20260709 12:12:12.489531 10478 leader_election.cc:403] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:12.440888 16291 consensus_queue.cc:1059] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:12:12.476783 16187 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:12.499977 16442 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:12:12.501621 16221 consensus_meta.cc:287] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.002s
I20260709 12:12:12.501698 16435 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.502776 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:12.474617 16445 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:12.516443 16446 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:12.517268 16446 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.517959 16435 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:12.487783 16383 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.513940 16442 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:12.591506 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:12.591778 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:12.594129 10505 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:12.612920 16446 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:12.613940 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 111286us
I20260709 12:12:12.587984 16445 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:12.615796 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:12.616132 16445 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.619076 16430 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:12.613082 15995 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 19. Current term is 20.
I20260709 12:12:12.629134 16353 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.629864 16353 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.595199 16442 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:12.624769 16444 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:12:12.672646 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.638841 16353 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:12.645536 16446 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.622364 16193 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.653903 16436 raft_consensus.cc:3055] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:12:12.663547 16438 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:12:12.668620 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.614554 16034 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.693321 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:12.721004 16443 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:12.727278 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 110062us
I20260709 12:12:12.695261 16436 raft_consensus.cc:740] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:12.708400 16313 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:12.730659 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:12.743047 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:12.736220 16448 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:12:12.738516 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:12.693434 16187 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.696851 16446 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.752776 11215 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:12.751014 16451 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.751188 11217 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 7 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:12.778923 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:12.786882 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 297711us
I20260709 12:12:12.787983 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:12.790391 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:12.797044 16431 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:12.797688 16431 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.798159 16431 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:12.753955 16313 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:12.811065 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 377006us
W20260709 12:12:12.827385 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 82827.9us
I20260709 12:12:12.835117 16187 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.819577 16456 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:12.836040 16456 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:12.836490 16456 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:12.837210 16452 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:12:12.840186 16034 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:12.817603 16451 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.843624 16451 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.824525 16425 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:12.855723 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65522.2us
I20260709 12:12:12.856076 16313 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.857911 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:12.827495 16436 consensus_queue.cc:260] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:12.860285 16436 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:12.863430 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:12.864228 11212 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:12.869489 16193 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.855615 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:12:12.887161 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 64784.7us
I20260709 12:12:12.891607 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:12.892513 10587 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:12.893249 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.898707 16034 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:12.899055 16187 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.899322 16034 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.899910 16187 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.900955 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.902112 16458 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:12.902393 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:12.893677 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.904908 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:12.911581 11214 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:12:12.904068 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52542.4us
I20260709 12:12:12.899911 16034 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:12.902750 16458 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:12.893229 16425 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:12:12.915019 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 57209.8us
W20260709 12:12:12.927057 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.927536 10513 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:12:12.932083 16446 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:12.932796 16446 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:12.909126 16428 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:12:12.882764 16305 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:12.915402 15847 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:12:12.954231 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.911085 16187 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:12.912497 16193 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:12.956058 16193 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.956678 16193 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:12.918308 16425 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:12.964967 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47632.9us
I20260709 12:12:12.987418 16443 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:12.992703 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 90468.9us
W20260709 12:12:12.981369 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 87782.8us
I20260709 12:12:12.970273 16279 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.009828 10513 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:13.014324 16383 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:13.015014 16383 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.017385 16383 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.018261 16383 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:12.991954 10587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:13.019580 10587 raft_consensus.cc:606] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.019959 10587 raft_consensus.cc:614] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:13.022468 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:13.023257 11217 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:13.027993 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:13.028004 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:12.980149 15847 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:13.029670 11214 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:12:13.035228 16450 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:13.035884 16450 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:13.036221 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:13.036412 16450 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:13.036902 10513 raft_consensus.cc:606] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.037314 10513 raft_consensus.cc:614] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:13.030717 16469 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:13.039337 16469 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.039840 16469 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:13.040917 10404 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:13.042008 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.044284 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:13.045660 11215 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:13.049609 16305 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.054903 15847 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.003995 16435 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:12:13.060246 16435 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:13.028573 16279 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.064064 16279 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.034518 16456 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.033877 16458 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.070354 10507 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:13.018682 16383 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.083928 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:13.043337 16446 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:12:13.087141 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:13.100060 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:13.073326 16456 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.110103 10435 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:12:13.110540 10435 pending_rounds.cc:85] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 9
I20260709 12:12:13.110798 10435 pending_rounds.cc:107] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:12:13.111114 10435 raft_consensus.cc:2889] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:13.112669 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:13.113694 10430 raft_consensus.cc:2410] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:12:13.089663 16313 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:13.119547 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:13.119541 16313 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.120421 10434 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:13.123241 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:13.128302 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82441.2us
W20260709 12:12:13.141362 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.148290 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:13.149191 10581 raft_consensus.cc:606] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.149672 10581 raft_consensus.cc:614] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:13.152854 10438 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:13.103400 16446 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:13.153156 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:13.162012 10435 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:13.165484 16450 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.167665 16193 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.169615 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:13.167733 10436 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:13.171782 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:13.178494 10438 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:13.179409 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:13.142251 16187 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:13.182264 10430 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:13.172688 16357 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:13.203265 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.204159 10513 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:13.146288 16442 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.187638 16187 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.192951 16449 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.213908 10434 raft_consensus.cc:1240] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:13.215159 11177 leader_election.cc:400] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:12:13.220082 11177 leader_election.cc:403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:13.216279 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:13.217792 10438 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:13.224537 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53344.3us
I20260709 12:12:13.199442 16313 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:13.183733 16435 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 12:12:13.227926 11178 leader_election.cc:336] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.803s (SENT)
I20260709 12:12:13.230078 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.217120 16451 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:13.217687 16187 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:13.241909 10434 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:13.222865 16436 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.222015 16443 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:13.246639 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.246876 10430 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:13.209053 16357 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.232374 16428 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:12:13.236186 16451 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.225880 16449 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:13.279444 16476 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:12:13.280797 16449 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:13.282519 16474 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:12:13.250921 16313 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:13.280824 10432 pending_rounds.cc:85] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 7
I20260709 12:12:13.292014 10432 pending_rounds.cc:107] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:12:13.292340 10432 raft_consensus.cc:2889] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:13.225579 16446 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.274542 16435 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:13.276479 16451 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:13.275275 16353 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:13.275818 16436 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:13.316149 16477 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:12:13.277263 16193 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:13.315763 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 112165us
I20260709 12:12:13.272073 16443 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:13.323963 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96076.8us
I20260709 12:12:13.242647 16425 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.292124 16313 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:13.283085 16431 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.313635 16435 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:13.338060 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 92922.4us
W20260709 12:12:13.335700 16467 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:12:13.329954 16313 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:13.345678 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:13.353225 11212 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
W20260709 12:12:13.347190 16479 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:12:13.330168 16450 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.352370 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.360343 10587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c882875c377649acb233035d47b8132e"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:13.361052 10587 raft_consensus.cc:606] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.363377 16279 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:13.365963 16279 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:13.366438 16279 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:13.368089 16436 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.350406 16442 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.374282 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:13.384514 16486 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:13.385146 16486 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.390626 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:13.391511 10586 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:12:13.385462 16431 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.399969 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:13.401413 16221 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:13.401950 16221 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.416994 16221 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:13.419540 16480 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:12:13.424333 16357 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:13.428344 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.431388 16449 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.432109 16449 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.433768 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:13.435647  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:13.424599 16425 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.431734 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:13.441668 11212 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:13.349804 16034 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.385571 16486 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:13.445662 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.446125 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:13.505743 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:13.506807 11217 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
W20260709 12:12:13.443005 16475 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:12:13.512506 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.514264 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:13.515045 10510 raft_consensus.cc:2376] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16. Re-sending same reply.
I20260709 12:12:13.441437 16449 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.516295 10505 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:13.435703 16442 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:13.518296 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.519908 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:13.528921 11212 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:12:13.533622 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:13.534497 10434 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:13.536743 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:13.539311 10434 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:13.383157 16435 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.550585 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:13.551339 10438 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:13.546504 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.549542 10430 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:13.552790 10512 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:12:13.562160 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.538955 16450 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:13.569412 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:13.570549 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:12:13.566344 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.574477 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.581581 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:13.601290  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:13.560822 16436 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:13.650640  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:13.651674 10433 raft_consensus.cc:1240] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:12:13.653651 16482 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.660077 10434 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:12:13.664184 16489 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:12:13.569954 15847 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:12:13.665309 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:13.666380 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104330us
I20260709 12:12:13.591506 16456 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:13.697369 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.721865 10505 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:12:13.723053 10507 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:13.704687 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:13.731329 10509 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:12:13.733318 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 164026us
I20260709 12:12:13.604609 16187 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:13.734325 16452 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:12:13.745385 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 305659us
W20260709 12:12:13.739564 16469 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.696s	user 0.017s	sys 0.000s
I20260709 12:12:13.750324 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.753469 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:13.775049 16494 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:12:13.777179 16034 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.713004 16436 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.781606 16436 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.748695 16456 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:13.757473 16469 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.803263 16187 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.807405 10505 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:13.827294 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 73894.9us
I20260709 12:12:13.828989 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:13.829810 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:13.829698 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:13.739328 16482 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.834828 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:13.811756 16456 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:13.836808 16456 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:13.836869 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89126.9us
I20260709 12:12:13.837989 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:13.838819 16491 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:12:13.839766 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 135225us
I20260709 12:12:13.840567 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:13.841584 15699 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:13.842011 15699 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:13.842433 15699 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:13.868211 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 134430us
I20260709 12:12:13.857378 16456 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.857519 16446 raft_consensus.cc:2764] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:13.876950 16505 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:13.875598 16436 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:13.876209 10512 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:12:13.867412 16430 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:12:13.865154 16422 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.829020 16279 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.825548 16313 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.915655 16482 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.907290 16353 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:13.919929 16353 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:13.921028 16353 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.923576 16519 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:13.924088 16519 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:13.925127 16519 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.926671 16519 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:13.928639 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 99811.2us
I20260709 12:12:13.912129 16486 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.933073 16313 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:13.933158 16486 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.936390 16442 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:13.939105 16507 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.299s	user 0.002s	sys 0.002s
I20260709 12:12:13.939426 16485 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:13.940086 16485 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.861092 16221 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:13.954461 11215 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:13.959932 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 122068us
I20260709 12:12:13.965123 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:13.950304 16291 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.983325 16291 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:13.985812 16291 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.988194 16383 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.991429 16456 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:13.994453 16485 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.995783 16443 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:13.996529 16221 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:14.000150 16451 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 12:12:14.007867 16451 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.003288 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:14.006515 16435 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:13.911895 16422 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.037679 16451 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.010382 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:13.982522 16469 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:14.013652 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 177466us
I20260709 12:12:14.043763 16469 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:14.044155 16497 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.315s	user 0.005s	sys 0.003s
I20260709 12:12:14.045812 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:13.979789 11220 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:12:14.022668 16446 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:14.016371 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:14.047816 16446 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:14.048187 16446 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:14.049082 16446 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:14.050127 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.050377 16446 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.051195 16503 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:14.051657 16503 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.052008 16503 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:14.052367 16458 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:14.053128 16279 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:14.057595 16357 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:12:14.060964 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:14.061595 11214 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:14.063771 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:14.065981 11213 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.067495 16469 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:14.071272 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.038621 16422 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.074477 16313 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:14.084582 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:14.029326 10438 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:14.080668 11214 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
W20260709 12:12:14.082163 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.092578 11213 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:12:14.095935 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:14.100631 10579 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.092443 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.095211 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:14.098279 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:14.106675 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:14.117996 10581 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:14.126089 10581 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:14.130378 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:14.131080 10580 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:14.109743 16451 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:14.133273 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 226143us
I20260709 12:12:14.129480 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.084468 16443 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:12:14.130098 16422 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.139379 15699 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.093369 16357 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:12:14.150802 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:14.149075 10512 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
W20260709 12:12:14.168599 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76250.7us
I20260709 12:12:14.147460 16458 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:14.209403 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52582.8us
I20260709 12:12:14.211627 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:14.207643 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:14.218300 10505 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:14.220144 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:14.223448 10580 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:14.177074 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.229918 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:14.230221 10331 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 112) took 17357 ms (client timeout 15000 ms). Trace:
I20260709 12:12:14.207118 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:14.200635 10553 leader_election.cc:336] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.507s (SENT)
I20260709 12:12:14.233024 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.231069 10586 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:14.236752 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:14.158342 16445 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:14.248345 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 71370us
I20260709 12:12:14.249517 10584 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:12:14.251194 16523 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:14.251677 16523 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.252017 16523 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:14.255174 10578 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:12:14.262276 16455 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:14.262936 16455 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.264072 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.263980 16455 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.158201 16430 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:12:14.265636 16299 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:12:14.266057 16299 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.266484 16469 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:14.167790 16383 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:14.169910 16431 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 12:12:14.268641 16383 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:14.207144 16521 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 12:12:14.269930 16431 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:14.264276 16519 consensus_queue.cc:1059] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:14.271879 16486 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:14.272765 16519 raft_consensus.cc:3055] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:14.272390 16431 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.271947 16485 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:14.280431 16485 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.281220 16431 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.285476 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:14.286499 16496 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:14.281558 16485 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.290753 16431 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:14.292833 15699 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.246850 16451 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:14.294920 16485 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.153729 16422 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.302277 11217 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:14.303474 16422 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.175828 16443 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.131s
I20260709 12:12:14.304636 10438 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:14.272106 16383 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.276810 16093 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:14.306330 16093 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.307340 16093 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.308346 16093 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.277944 16313 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:14.310075 16313 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:14.311153 16313 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.312988 16313 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.267095 16299 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.317057 16299 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:14.273276 16519 raft_consensus.cc:740] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.319829 16519 consensus_queue.cc:260] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.278728 16521 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:14.278800 16515 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:12:14.322763 16515 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:14.323235 16515 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.324110 16515 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:14.230700 10331 rpcz_store.cc:269] 0709 12:11:56.873054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.880963 (+  7909us) service_pool.cc:224] Handling call
0709 12:11:57.168998 (+288035us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b5400270010. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb0b2 00007f2c98934759 00007f2c988cca76 00007f2c9889dd58 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc
0709 12:11:58.726431 (+1557433us) spinlock_profiling.cc:243] Waited 42.2 ms on lock 0x7b54003666a8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c98cd1331 00007f2c98ccc3fc 00007f2c9889de84 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc 00007f2c96e5c392 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:14.230130 (+15503699us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:12.511763 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:12.527906 (+ 16143us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:12.527994 (+    88us) write_op.cc:435] Acquired schema lock
0709 12:12:12.528023 (+    29us) tablet.cc:662] Decoding operations
0709 12:12:12.599495 (+ 71472us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:12.623336 (+ 23841us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:12.623393 (+    57us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:12:12.625181 (+  1788us) tablet.cc:701] Row locks acquired
0709 12:12:12.625245 (+    64us) write_op.cc:260] PREPARE: finished
0709 12:12:12.625476 (+   231us) write_op.cc:270] Start()
0709 12:12:12.625676 (+   200us) write_op.cc:276] Timestamp: P: 1783599132625396 usec, L: 0
0709 12:12:12.625722 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 12:12:12.679178 (+ 53456us) log.cc:844] Serialized 4046 byte log entry
0709 12:12:13.024633 (+345455us) op_driver.cc:464] REPLICATION: finished
0709 12:12:13.089931 (+ 65298us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c8f9d50ac 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c96515ef0
0709 12:12:13.246195 (+156264us) write_op.cc:301] APPLY: starting
0709 12:12:13.271573 (+ 25378us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:13.395998 (+124425us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:13.396056 (+    58us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:13.652572 (+256516us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:13.652954 (+   382us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:12:13.653018 (+    64us) write_op.cc:312] APPLY: finished
0709 12:12:13.653637 (+   619us) log.cc:844] Serialized 97 byte log entry
0709 12:12:13.653890 (+   253us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:13.654060 (+   170us) write_op.cc:454] Released schema lock
0709 12:12:13.654140 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:13.654627 (+   487us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:13.654674 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":614454,"reported_tablets":18,"spinlock_wait_cycles":129041024,"child_traces":[["op",{"apply.queue_time_us":196790,"apply.run_cpu_time_us":21088,"apply.run_wall_time_us":422558,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":224174,"prepare.run_cpu_time_us":38275,"prepare.run_wall_time_us":334328,"raft.queue_time_us":153373,"raft.run_cpu_time_us":41047,"raft.run_wall_time_us":165257,"replication_time_us":398688,"spinlock_wait_cycles":5838720,"thread_start_us":514211,"threads_started":4,"wal-append.queue_time_us":142691}]]}
I20260709 12:12:14.326853 11214 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.327494 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.328720 16451 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:12:14.332971 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.341379 10579 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:14.343760 10438 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:12:14.347723 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.348492 10438 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:14.351897 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:14.352779 10579 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:14.348869 10403 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.356187 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:14.356889 10437 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:14.357858 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:14.358199 10438 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:14.358541 10578 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:14.362543 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:14.366721 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:14.368887 10435 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:14.369325 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.370115 10433 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
I20260709 12:12:14.375309 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.376097 10431 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:14.376971 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.378391 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.377015 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:14.379127 10478 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:14.379307 10578 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:14.379231 10436 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:14.381124 16521 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:12:14.385283 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 236986us
I20260709 12:12:14.395727 11217 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:12:14.286576 16503 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.352813 16515 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.390620 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:12:14.402666 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 202132us
W20260709 12:12:14.404917 10553 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.552s (SENT)
I20260709 12:12:14.405349 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.409780 16504 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:14.410442 16504 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:14.361775 16519 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:14.411731 16504 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.412647 16503 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:14.409623 10330 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 6 to 9, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 9 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:14.427703 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:14.428791 10429 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:14.395846 16451 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:14.432224 16520 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:12:14.432819 10513 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:14.447616 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 104152us
W20260709 12:12:14.452010 16528 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:12:14.457746 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.474778 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70418.3us
I20260709 12:12:14.466679 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:14.478943 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51935.9us
I20260709 12:12:14.489611 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:14.522197 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117310us
W20260709 12:12:14.523113 10553 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.609s (SENT)
I20260709 12:12:14.490461 16503 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:14.523612 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.490859 16521 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:12:14.523375 16487 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:14.505338 16486 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:12:14.511215 16357 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 12:12:14.561314 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.560711 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:14.547408 16490 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:12:14.569077 10553 leader_election.cc:336] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.669s (SENT)
I20260709 12:12:14.549925 16521 consensus_queue.cc:1243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:12:14.585287 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.563050 16487 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:14.591962 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:12:14.599882 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 118000us
I20260709 12:12:14.589143 16445 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:14.604861 10553 leader_election.cc:336] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.697s (SENT)
I20260709 12:12:14.593650 16456 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.610440 10578 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:12:14.594864 16451 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:12:14.648267 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:14.642599 16527 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:12:14.673684 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.675990 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:14.678375 15699 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:14.681705 16523 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.676093 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:14.607439 16445 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:14.687031 10513 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.705749 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:14.706594 10505 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:14.707083 16533 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:12:14.708971 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:14.706671 16532 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:12:14.711647 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106804us
W20260709 12:12:14.712577 10553 leader_election.cc:336] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.742s (SENT)
I20260709 12:12:14.674490 16357 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:12:14.720712 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.722360 16535 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:14.723466 16482 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:14.724716 16524 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:14.725121 16524 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.726048 16524 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.727627 16524 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.727573 16445 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.729072 16445 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:14.731360 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:14.732743 16486 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:14.733151 16486 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:14.734078 16486 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:14.735625 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.736219 16486 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:14.736495 15847 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:14.736965 15847 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.737341 15847 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:14.740715 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.741585 10478 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:14.690846 15699 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.759055 10429 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
W20260709 12:12:14.764318 16530 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 12:12:14.647476 16456 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.768857 16456 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:14.769436 16093 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:14.769894 16093 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.766448 15847 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.770284 16093 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:14.722839 16535 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.775563 16535 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:14.777130 16527 log.cc:927] Time spent T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:12:14.718001 16431 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:12:14.782387 15699 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.783727 15699 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:14.783783 16279 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:14.784423 16279 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.784940 16520 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:14.719013 16529 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:12:14.707872 16455 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:14.793162 10505 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
W20260709 12:12:14.796698 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.719431 16451 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:14.797399 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:14.799226 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:14.812256 10513 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:14.719305 16469 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:14.816277 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:14.831596 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.824019 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 232193us
I20260709 12:12:14.846239 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.847168 10429 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
W20260709 12:12:14.847992 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 135464us
I20260709 12:12:14.797326 16455 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.797137 16357 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:12:14.824978 16458 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.006s
I20260709 12:12:14.871320 16458 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:14.807951 16451 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.872354 16451 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.872731 16093 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.874682 16541 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:14.836910 16469 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.875334 16541 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.877434 16523 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.880744 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:14.881455 16504 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:14.882069 16504 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:14.853237 16279 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.829967 16485 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:14.884637 16485 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:14.885064 16485 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.885154 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:14.886240 16523 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:14.887917 16523 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:14.891902 16093 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:14.892740 16541 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:14.858549 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.892879 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:14.893776 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.753018 16487 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:14.894932 10553 leader_election.cc:336] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.882s (SENT)
I20260709 12:12:14.895406 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.896195 16373 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:14.854666 15995 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:12:14.830113 16435 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.884079 16422 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:14.817649 16515 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:12:14.899134 10507 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:12:14.856891 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:14.924870 11220 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:14.925369 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:14.901731 10437 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.001s
I20260709 12:12:14.877540 16469 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:14.910488 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.923900 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:14.926295 10437 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:14.926692 10436 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:14.896735 16373 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.929898 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:14.933529 10433 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:14.857896 16455 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.938197 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:14.961155 10431 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:12:14.963685 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.908330 16503 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:14.927966 16442 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:14.979092 16373 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:14.980991 16535 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:14.999378 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:14.967294 16422 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:14.983235 16504 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:15.006475 10553 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.566s (SENT)
I20260709 12:12:15.016964 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:14.929255 16435 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:15.014557 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.020743 16435 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:15.003520 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.022032 16436 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:15.023623 10507 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:15.025642 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.026580 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.016309 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:15.027534 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.028407 10580 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:12:15.029176 10512 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:15.039158 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.018097 16519 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:15.043965 10507 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:12:15.047256 10553 leader_election.cc:336] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.584s (SENT)
I20260709 12:12:15.047969 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.026369 16383 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:15.058203 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:15.059377 11217 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:15.066556 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:15.068017 10553 leader_election.cc:336] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.833s (SENT)
I20260709 12:12:15.068410 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.069324 10580 raft_consensus.cc:2410] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:15.074221 10436 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:15.070928 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.044755 16520 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:15.075943 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:15.067854 16503 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:15.084007 10553 leader_election.cc:336] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.938s (SENT)
I20260709 12:12:15.071866 16519 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.084787 16447 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:15.094180 16422 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.069118 16221 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:12:15.116624 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.124298 10505 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:15.125401 10580 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:15.126744 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.127545 10580 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:12:15.095028 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.120882 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.133147 10437 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:15.134712 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.135492 10586 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:15.070310 16504 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:12:15.084684 15847 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.093237 16520 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:12:15.141582 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61412.2us
I20260709 12:12:15.142249 16503 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:15.143849 16535 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.151202 10433 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:15.099472 16447 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.154911 10553 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:15.158172 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.095057 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:15.094818 16546 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:15.167600 16520 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.168946 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.170614 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:15.212366 10513 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:15.171545 10506 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:15.180188 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.213472 10433 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:15.165621 16456 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:12:15.186509 11220 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:15.187302 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.188194 16519 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.188937 16485 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.189994 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.190338 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.191807 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.192313 16279 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:15.193933 16524 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:15.177881 10553 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.082s (SENT)
I20260709 12:12:15.106514 16451 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.200449 11217 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:12:15.200786 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 107063us
I20260709 12:12:15.180440 10580 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:15.203166 10586 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:15.137809 16455 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.221359 10581 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:12:15.241232 10438 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:12:15.241441 10433 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:15.242817 10585 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.209365 16435 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:15.202891 16546 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:15.210728 16548 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:15.259299 10479 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:15.239537 16313 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:15.272457 10431 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.281550 10581 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:15.247171 16524 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.287202 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:15.289920 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 112081us
I20260709 12:12:15.259764 16435 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:15.309417 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.271631 16546 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:15.311818 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62202.8us
I20260709 12:12:15.271211 16313 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:15.254666 16445 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:12:15.259512 16521 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:12:15.324061 10553 leader_election.cc:336] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.125s (SENT)
I20260709 12:12:15.326637 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.327181 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:15.328996 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69821.9us
I20260709 12:12:15.331949 10582 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:15.256526 16279 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.308594 16435 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:15.336361 16373 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:15.346709 10582 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:15.347889 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.324312 16313 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:15.351320 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 86297.7us
I20260709 12:12:15.351760 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.326516 16445 consensus_queue.cc:1243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 96a7b6a7b75246d381f5d4a71ea716d9 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:15.403934 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.405705 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.405975 10506 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:15.408556  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:15.414635 10585 raft_consensus.cc:2468] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:15.416267 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.417075 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:15.420289 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96289.2us
W20260709 12:12:15.421841 10553 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.853s (SENT)
I20260709 12:12:15.422313 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.276403 16447 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.425474 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:15.403784 16538 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:12:15.441900 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.444268 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.408372 16536 raft_consensus.cc:2804] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:15.449569 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.454325 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.456017 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:15.454461 10553 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.144s (SENT)
I20260709 12:12:15.456792 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.424749 16469 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.440209 16446 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.456684 10582 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 8.
I20260709 12:12:15.441463 16383 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:15.463276 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.472891 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.473171 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:15.475745 10437 raft_consensus.cc:1240] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:15.476164 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.458864 16436 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:15.481640 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.485963 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:15.486265 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.491744 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:15.495420 16555 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:12:15.496235 10437 raft_consensus.cc:1240] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:15.479183 16536 raft_consensus.cc:697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:15.499017 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44577.7us
I20260709 12:12:15.498762 16536 consensus_queue.cc:237] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:15.499854 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.181s (SENT)
I20260709 12:12:15.500314 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.501888 16490 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:15.462262 16541 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:15.501853 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.505007 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.506906 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:15.410651 16524 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.473318 16487 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:15.520905 10506 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:12:15.522080 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.524549 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.525336 10582 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:12:15.485009 16485 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:15.545411 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.547533 10429 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:15.553222 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.558065 10505 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:15.470870 16469 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:15.564894 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.563201 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.566433 10505 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:15.567729 10512 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:15.569589 10507 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:12:15.573837 16445 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:15.596072 16445 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:15.597275 16445 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.573639 16443 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:15.600004 16443 consensus_queue.cc:1243] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:12:15.600630 16443 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:15.600970 16443 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:15.566346 16548 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:15.602126 16548 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.574801 16383 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:15.575536 16560 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:15.575717 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.516979 16451 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.607722 16560 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:15.473822 16446 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.608189 16560 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:15.609205 16560 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.609915 16446 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:15.610695 10438 raft_consensus.cc:2376] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13. Re-sending same reply.
I20260709 12:12:15.611253 16523 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:15.611726 16523 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.613103 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:15.613898 11220 raft_consensus.cc:2393] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:15.617002 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.624902 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.626130 10434 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.628784 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.532007 16455 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.633160 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.634536 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.648471 16279 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.651065 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.673206 10584 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.662735 10433 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:15.674077 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 52722.9us
I20260709 12:12:15.518358 16541 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.669469 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:15.676546 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.672307 10437 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.674738 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.683339 10434 raft_consensus.cc:2393] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.688769 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.689790 10578 raft_consensus.cc:2410] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:15.665550 16560 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.699430 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.700320 10437 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.703190 16548 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.677691 16443 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.708484 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.709776 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.711369 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.715967 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.716717 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:15.717935 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:15.719851 16455 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:15.720456 16455 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.721452 16455 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.723683 15699 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:15.724539 15699 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.725822 15699 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.694396 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.727346 16313 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.728616 10585 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:15.728760 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.729182 16469 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:15.729498 10580 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:15.730274 10553 leader_election.cc:336] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.569s (SENT)
I20260709 12:12:15.735978 16313 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:15.738283 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:15.739204 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.740438 10433 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:15.699431 10434 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:12:15.743165 16541 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:15.745205 10553 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.634s (SENT)
I20260709 12:12:15.745834 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.747206 16357 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:15.749202 10512 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:15.750278 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.751345 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.752213 10505 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:12:15.753067 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.754607 10504 pending_rounds.cc:85] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 12
W20260709 12:12:15.754822 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.754936 10504 pending_rounds.cc:107] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:12:15.756125 10504 raft_consensus.cc:2889] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:15.757315 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.758329 10512 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.759016 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.759335 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.710812 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:15.760516 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:15.760722 10587 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:15.739549 10507 raft_consensus.cc:606] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.780349 10510 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:15.781785 10584 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:15.785214 10583 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: []
W20260709 12:12:15.789532 10553 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.732s (SENT)
I20260709 12:12:15.790186 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.794111 10582 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 10.
I20260709 12:12:15.797382 10578 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:12:15.803828 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.810585 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:15.807147 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.813076 10580 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:15.815374 10437 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.816411 10435 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:12:15.829893 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.833437 10513 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:15.834679 10432 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:15.799769 16469 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:15.832237 16485 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:15.812225 16548 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:12:15.887900 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.891098 10432 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.836153 16490 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:12:15.885040 16445 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:15.881096 16486 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:12:15.904537 10508 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:15.907456 10585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:15.893129 16485 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:15.909320 16485 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.912884 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.913828 10435 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.913852 16383 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.915081 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.914873 16383 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.916323 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.916936 10435 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:15.917809 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.919360 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.923645 10434 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:15.924803 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:15.924494 10580 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 7 to 10, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:15.925594 16383 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:15.927351 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.927444 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.928925 10433 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:15.929986 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.930763 10431 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:15.931267 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:15.931751 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.932159 10582 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:12:15.932577 10429 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:12:15.933380 16435 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.008s
I20260709 12:12:15.933511 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.933869 16435 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:15.935760 10579 raft_consensus.cc:1813] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Handling vote request from an unknown peer 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.936321 10579 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17.
W20260709 12:12:15.940558 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.762s (SENT)
I20260709 12:12:15.941207 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.940547 10587 raft_consensus.cc:2410] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:12:15.942971 10585 raft_consensus.cc:606] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:15.943607 10585 raft_consensus.cc:614] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:15.950277 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.930543 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.952730 10432 raft_consensus.cc:2368] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:12:15.918853 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:15.955658 10435 raft_consensus.cc:2368] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:15.851325 16469 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:15.970070 10437 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
W20260709 12:12:15.975085 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.939577 16435 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:15.965685 10434 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:12:16.005393 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.006747 10435 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:12:16.008895 10433 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:16.011538 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.011094 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:16.019533 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 125264us
W20260709 12:12:16.020323 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.969331 16581 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:15.945013 16445 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.024073 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.026753 10434 raft_consensus.cc:2393] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:12:16.028318 10553 leader_election.cc:336] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.978s (SENT)
I20260709 12:12:16.028863 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.030505 10435 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:16.032162 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.032925 10435 raft_consensus.cc:2393] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.037539 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.038479 10437 raft_consensus.cc:2410] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:12:16.040657 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.059123 10435 raft_consensus.cc:2368] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:12:16.060283 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:16.064138 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89657.6us
I20260709 12:12:16.060652 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:12:16.062287 10553 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.012s (SENT)
W20260709 12:12:16.067634 10553 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.013s (SENT)
I20260709 12:12:16.068487 10432 raft_consensus.cc:2368] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19.
I20260709 12:12:16.035329 16515 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.054431 16435 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:16.079916 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.082327 16436 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:16.083909 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.084451 15995 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:16.085583 16561 raft_consensus.cc:993] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:16.087682 16445 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:16.088780 16469 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:16.041164 16535 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.122308 10435 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:12:16.099571 16543 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.026s	user 0.010s	sys 0.000s
I20260709 12:12:16.106596 16353 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index 7 to 10, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:16.111042 10553 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.051s (SENT)
I20260709 12:12:16.120508 16574 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.116987 10430 raft_consensus.cc:2376] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18. Re-sending same reply.
I20260709 12:12:16.141397 10507 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:12:16.122849 16577 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:12:16.147264 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.147514 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.147296 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.153774 10508 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:16.149611 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:16.140532 16578 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:12:16.157339 10435 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.157729 10509 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38166 (request call id 666) took 1437 ms. Trace:
I20260709 12:12:16.158188 10509 rpcz_store.cc:276] 0709 12:12:14.720373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.720569 (+   196us) service_pool.cc:224] Handling call
0709 12:12:14.761207 (+ 40638us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:14.770826 (+  9619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:14.770861 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:14.771495 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:12:14.904158 (+132663us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:14.959363 (+ 55205us) log.cc:844] Serialized 14 byte log entry
0709 12:12:14.959749 (+   386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.959864 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:15.580875 (+621011us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b18000a44a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1
0709 12:12:15.664195 (+ 83320us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b18000a44a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:16.098888 (+434693us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c9267a86e 00007f2c92679bb6 00007f2c926797d8 00007f2c92683221 00007f2c926831d9 00007f2c92683169
0709 12:12:16.157496 (+ 58608us) raft_consensus.cc:1685] finished
0709 12:12:16.157523 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.157665 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9168,"prepare.queue_time_us":58895,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":20243,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":13642,"spinlock_wait_cycles":55186816,"thread_start_us":132161,"threads_started":1,"wal-append.queue_time_us":258480}
I20260709 12:12:16.158965 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.162559 10435 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.163744 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.165324 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.166378 10435 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.139825 16558 raft_consensus.cc:1081] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:16.142634 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.167747 10432 raft_consensus.cc:2376] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20. Re-sending same reply.
I20260709 12:12:16.167671 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.138154 16541 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:16.174777 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 366211us
I20260709 12:12:16.176172 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:16.181713 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.118022 16515 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.182142 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:16.182749 10513 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:16.186278 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.186939 10512 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:16.188053 10582 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:12:16.170838 16455 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.210937 16455 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.215423 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.216100 10432 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:16.217505 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:16.218214 11220 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:16.175156 16435 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.219461 16435 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.177338 16541 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.221887 16435 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:16.222630 16541 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:16.208606 16469 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:16.145972 16535 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.175377 16569 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:12:16.197041 10513 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:12:16.224706 16560 consensus_meta.cc:287] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.001s
W20260709 12:12:16.249146 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 55322.7us
I20260709 12:12:16.196508 16581 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.174525 16373 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:16.275954 16556 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 1.012s	user 0.008s	sys 0.000s
I20260709 12:12:16.278115 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.278448 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.280393 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.280969 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.281883 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.282253 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.224820 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.289852 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.291261 10579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 2784) took 1156 ms. Trace:
I20260709 12:12:16.282949 10438 raft_consensus.cc:2368] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18.
I20260709 12:12:16.302534 10509 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:12:16.308394 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.235750 16583 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:12:16.239168 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 128188us
I20260709 12:12:16.254048 10510 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:12:16.257563 16469 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.260802 16523 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.267683 16535 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:16.270303 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:16.260926 16515 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:16.346622 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76416.1us
I20260709 12:12:16.347335 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.347594 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:16.278919 10433 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:16.348906 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.304165 10579 rpcz_store.cc:276] 0709 12:12:15.135271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.190099 (+ 54828us) service_pool.cc:224] Handling call
0709 12:12:16.290999 (+1100900us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54001a13d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:16.291162 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593331456}
W20260709 12:12:16.230495 16546 consensus_meta.cc:287] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.899s	user 0.012s	sys 0.000s
I20260709 12:12:16.214217 16455 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:16.275701 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:16.352833 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.353089 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.356215 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.356987 10430 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.353044 11217 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:16.362941 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.369297 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.370843 10430 raft_consensus.cc:2393] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.373183 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.374002 10433 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.353952 16373 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:12:16.378113 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.283661 16581 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.390599 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.385587 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.391673 10438 raft_consensus.cc:2368] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:16.392016 10433 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:16.378578 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.361955 16445 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:16.389912 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:16.382193 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.768s (SENT)
I20260709 12:12:16.397763 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.392699 10430 raft_consensus.cc:2376] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18. Re-sending same reply.
I20260709 12:12:16.392874 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.399451 10433 raft_consensus.cc:2376] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20. Re-sending same reply.
I20260709 12:12:16.346622 16560 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:12:16.400404 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.401271 10431 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.404062 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.357674 16546 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.409444 10430 raft_consensus.cc:2368] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:12:16.391400 16373 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:16.410897 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.407687 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.438793 10433 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.408138 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.414019 10432 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:16.376106 16515 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:12:16.440419 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.441565 10431 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.441776 10438 raft_consensus.cc:2393] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.443723 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.444458 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.445319 10433 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.407569 16445 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:16.440907 16373 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.452934 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.456308 16546 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:16.459777 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:16.464325 10431 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:16.465034 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.465696 10430 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:16.464352 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:16.468222 10506 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17.
I20260709 12:12:16.469216 16581 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:16.469712 11177 leader_election.cc:400] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17.
I20260709 12:12:16.470031 11177 leader_election.cc:403] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:12:16.473752 16569 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:16.518492 10438 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:16.463888 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.520453 10431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:16.521219 10431 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:16.519414 10433 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:16.521675 10431 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:16.522467 10430 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:16.461447 16520 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:16.532209 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 216166us
I20260709 12:12:16.532754 10432 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:16.537720 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:16.534847 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:16.463461 16568 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:12:16.523674 16590 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
I20260709 12:12:16.542462 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:16.566133 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.592566 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:16.593510 10432 raft_consensus.cc:2393] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:16.593866 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.595160 10513 raft_consensus.cc:2410] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 12:12:16.604930 16593 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:12:16.605997 16589 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:12:16.536032 16585 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:12:16.605026 16594 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:12:16.628530 10432 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:16.617975 16469 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:16.626025 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 243860us
W20260709 12:12:16.640125 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.603708 16353 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33248 (request call id 10) took 2377 ms. Trace:
I20260709 12:12:16.671404 16353 rpcz_store.cc:276] 0709 12:12:14.217218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.217387 (+   169us) service_pool.cc:224] Handling call
0709 12:12:14.282330 (+ 64943us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b5400791420. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a5c520 00007f2c92ae5b42 00007f2c92ae4800 00007f2c92add9fa 00007f2c97aa54c9 00007f2c91021c54 00007f2c91021be2 00007f2c91021b11 00007f2c91021a8c 00007f2c91020d42
0709 12:12:14.358662 (+ 76332us) log.cc:844] Serialized 487 byte log entry
0709 12:12:14.676708 (+318046us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b1800373300. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c92a38b0b 00007f2c92abdb63 00007f2c92ae5752 00007f2c92add9fa
0709 12:12:16.594545 (+1917837us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":676,"mutex_wait_us":174122,"raft.queue_time_us":183665,"raft.run_cpu_time_us":1466,"raft.run_wall_time_us":7484,"spinlock_wait_cycles":100959744,"thread_start_us":236854,"threads_started":1,"wal-append.queue_time_us":223281,"wal-append.run_cpu_time_us":3212,"wal-append.run_wall_time_us":1296374}
I20260709 12:12:16.648923 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:16.675220 10553 leader_election.cc:336] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 2.331s (SENT)
I20260709 12:12:16.675873 16435 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.676469 16435 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.676913 16541 raft_consensus.cc:2764] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:12:16.678112 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:16.679006 16541 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:16.633198 16585 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.681200 16585 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:16.681988 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet deab11eaf55b4f3ebe48926a1e2afa38 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:16.591621 16568 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.544281 16445 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.675822 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.652339 10432 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:16.668145 16523 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:16.692799 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:12:16.705049 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57847.8us
I20260709 12:12:16.678684 16435 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:16.717751 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:16.702638 10431 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.721571 10433 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:16.724102 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:16.708683 16446 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:16.708498 16357 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:16.725369 16560 consensus_queue.cc:1059] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
W20260709 12:12:16.742237 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67055us
I20260709 12:12:16.719432 16469 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:16.729463 16451 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:12:16.780591 16577 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:12:16.807180 10432 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:16.807796 10432 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:16.811964 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87959.6us
I20260709 12:12:16.735212 16560 consensus_queue.cc:1059] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:12:16.832870 10553 leader_election.cc:400] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:16.813726 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:16.834048 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.838028 10553 leader_election.cc:403] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:12:16.833307 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 109512us
W20260709 12:12:16.853335 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.853113 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47956.5us
I20260709 12:12:16.835040 10432 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.855546 10432 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.856015 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.802173 16422 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:16.806255 16541 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:16.859434 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:16.859840 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46207.2us
I20260709 12:12:16.864012 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.864435 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.866595 10513 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:16.866786 10431 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:16.868270 10553 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.839474 16603 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:16.795512 16469 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.868498 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:16.869792 16603 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:16.871977 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.872278 16560 raft_consensus.cc:3055] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:16.872713 16560 raft_consensus.cc:740] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:16.872427 16603 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:16.873996 16603 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:16.873762 16560 consensus_queue.cc:260] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:16.874606 16605 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:12:16.875298 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:16.875559 16560 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:16.878120 16469 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:16.832211 16585 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:12:16.861294 16357 raft_consensus.cc:3060] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:16.863248 16435 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:16.867902 16599 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:16.884140 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:16.884557 16587 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:16.888186 16613 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:16.892409 16569 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17.
I20260709 12:12:16.893347 16541 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:16.867092 16422 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:16.913843 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 62178.7us
I20260709 12:12:16.916109 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:16.923341 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54938.9us
I20260709 12:12:16.924746 10403 leader_election.cc:400] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17.
I20260709 12:12:16.925808 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.926707 16451 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:16.927498 16451 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:16.929204 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.930915 16616 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:16.932654 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.933477 16616 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.934850 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.936697 16616 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:16.938423 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.939507 16616 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:16.941198 16373 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.942498 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:16.944358 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.944976 16443 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.945297 16616 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.945848 16373 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:16.946748 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.947664 16357 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:16.948118 16616 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.949815 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:16.953541 16616 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:16.956394 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:16.957656 16568 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:16.958655 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:16.960637 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.963316 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47327.8us
W20260709 12:12:16.961171 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117321us
I20260709 12:12:16.972551 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:17.013670 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.036258 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.039440 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.118690 16584 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:17.119232 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.119576 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:17.119943 10436 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:16.919514 16613 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:16.918136 16435 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:17.004453 16581 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:12:17.111742 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.014377 16373 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 12:12:17.115283 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 146429us
W20260709 12:12:17.127020 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 173520us
I20260709 12:12:17.004112 16616 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:17.122696 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 156946us
I20260709 12:12:17.115113 16490 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:17.128129 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:12:17.123853 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70498.5us
I20260709 12:12:16.988528 10580 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:17.117053 10509 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:17.133949 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.138621 10580 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:17.111301 16591 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:17.136915 10479 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:17.137799 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.123518 16584 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.135488 16568 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.126362 16541 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:17.157512 11178 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.148931 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.158804 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.148222 10585 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:17.158829 10578 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:17.160073 16279 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:17.151306 16447 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:12:17.165599 16564 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:17.166105 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:17.154397 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.169008 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:17.145151 10581 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:17.172345 16490 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:17.152631 16560 consensus_queue.cc:1059] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:12:17.173058 16490 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.162853 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.140419 16357 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:17.175518 10583 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:17.176137 10510 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 677) took 1054 ms. Trace:
I20260709 12:12:17.176530 10510 rpcz_store.cc:276] 0709 12:12:16.121882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:16.122157 (+   275us) service_pool.cc:224] Handling call
0709 12:12:17.175948 (+1053791us) spinlock_profiling.cc:243] Waited 99.8 ms on lock 0x7b5400229cd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:17.176048 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229650560}
I20260709 12:12:17.167508 16604 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.179701 16490 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:17.144668 16619 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:17.187867 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:17.188613 10511 raft_consensus.cc:606] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.142874 16445 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:17.188387 16619 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:17.189796 16445 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.190797 16619 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.153667 16558 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:17.143158 16451 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:17.183972 16291 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:17.175531 16357 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.142969 16446 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.166136 16564 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.154699 16599 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:12:17.144302 16469 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.170426 10512 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:17.192425 16445 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:17.152510 16568 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.133077 16569 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:17.236997 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:17.229362 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61927.4us
I20260709 12:12:17.145599 16584 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.141386 16591 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:17.244776 15847 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:17.270444 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 142441us
I20260709 12:12:17.244315 16451 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.230850 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:17.259023 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.272668 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:17.241904 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.251839 16564 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:17.275425 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.277000 10585 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:17.278805 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.280306 16615 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:17.280772 16615 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:17.239873 16585 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:12:17.238603 16558 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.154063 16541 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.247999 16446 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.290679 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.295120 16563 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:12:17.270835 15847 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.296375 15847 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:17.296554 16613 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.296708 16446 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:17.294801 16523 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.000s
I20260709 12:12:17.275257 15995 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:12:17.300814 16621 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:12:17.302450 16617 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:17.311164 16600 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:17.313494 16279 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:17.315207 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 2791) took 1566 ms. Trace:
I20260709 12:12:17.315657 10504 rpcz_store.cc:276] 0709 12:12:15.748653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.753795 (+  5142us) service_pool.cc:224] Handling call
0709 12:12:15.754229 (+   434us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:15.756855 (+  2626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:15.756889 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:16.687626 (+930737us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b2c0001c5e8. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:12:16.688264 (+   638us) log.cc:844] Serialized 52 byte log entry
0709 12:12:16.917447 (+229183us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:16.917497 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.917596 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:17.304822 (+387226us) raft_consensus.cc:1685] finished
0709 12:12:17.304847 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:17.315116 (+ 10269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1251,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":41088768,"thread_start_us":228827,"threads_started":1,"wal-append.queue_time_us":443858}
I20260709 12:12:17.290668 16535 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:12:17.316376 15847 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.321019 16535 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:17.321486 16535 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:17.322666 15847 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.281411 16445 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:17.324231 16445 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.326586 16445 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:17.328544 16622 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:12:17.329188 15699 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:17.329753 15699 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.331365 16591 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.331774 16567 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:17.332587 16591 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.332041 15699 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.335889 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:17.339524 16451 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.339783 10582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:17.340062 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.340662 10582 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.341102 10582 raft_consensus.cc:614] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:17.341212 10587 raft_consensus.cc:1275] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:17.341693 10580 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:17.343324 16625 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:17.344312 16617 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:17.344902 10504 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:17.345726 10504 raft_consensus.cc:606] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.346086 10504 raft_consensus.cc:614] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:17.345417 16617 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.349937 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.350819 11217 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:17.353274 16616 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:12:17.353922 16616 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:17.355068 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.354832 16616 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.356598 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.353369 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:17.258164 16469 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.371840 10432 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:12:17.372939 16469 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.374115 16568 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.374193 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.375416 10504 raft_consensus.cc:2393] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 10: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:17.380663 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.385711 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:17.358892 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74445.3us
I20260709 12:12:17.388928 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.360466 16612 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.360929 16521 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:17.363374 10430 pending_rounds.cc:85] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:12:17.396385 10430 pending_rounds.cc:107] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:12:17.396745 10430 raft_consensus.cc:2889] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:17.268811 16569 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.364660 16608 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:17.366139 16357 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:17.395254 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 121777us
I20260709 12:12:17.394857 10585 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:17.406422 10403 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.407097 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.419961 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.425593 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:17.425940 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.426517 10437 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:17.362886 16558 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.390924 16612 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:17.440411 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:17.401847 16584 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.402493 16608 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.444511 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.444691 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:17.451088 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:17.322573 16535 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.424508 16451 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:17.461608 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.424244 16625 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:17.407332 16485 consensus_queue.cc:1048] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:12:17.475210 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:17.404604 16520 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:12:17.470041 10437 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:17.439471 16373 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.440969 16612 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.462407 16608 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:17.476265 11213 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:12:17.480656 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.503669 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.481854 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.484377 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.484916 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.485711 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.507265 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:17.507866 10508 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:17.470458 16469 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:17.467590 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:17.416989 16563 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:12:17.442714 16626 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:17.490022 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.492985 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.452512 16613 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:17.473937 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:17.496191 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.497027 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:17.497846 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.403949 16564 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.545996 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.490128 16443 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:17.470301 16561 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.465253 16585 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:17.487731 10431 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:12:17.511247 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.557824 10437 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:17.559111 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.511330 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:12:17.468119 16482 consensus_queue.cc:1175] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 16 status { last_received { term: 16 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 16 index: 9 } }
I20260709 12:12:17.530011 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.539044 10434 raft_consensus.cc:2376] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17. Re-sending same reply.
I20260709 12:12:17.562692 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.539103 10583 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.565349 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:17.566413 10434 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:17.498596 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:17.487788 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:17.547772 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76559.8us
I20260709 12:12:17.499415 16373 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.462622 16584 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.513043 16469 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.515861 11213 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:17.543936 16620 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:17.603681 16620 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.606590 16620 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.548427 10432 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:17.608317 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.609745 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.577121 10587 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:17.573884 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:17.613121 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.614439 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:17.616387 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.580471 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:17.513419 16490 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:12:17.556916 16613 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:17.533730 16357 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:12:17.596592 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:17.599714 16564 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.680442 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:17.582463 16617 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:12:17.575477 16430 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:17.683030 10429 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:17.682879 16430 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.684134 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:17.575865 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 64600.2us
W20260709 12:12:17.687844 16601 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.894s	user 0.002s	sys 0.008s
I20260709 12:12:17.687024 16612 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.610913 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.633971 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.636112 16616 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:17.643791 16596 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:17.691457 16612 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:17.691946 16596 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:17.692565 16596 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.617128 16541 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:17.694756 10434 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
W20260709 12:12:17.654273 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.696273 10434 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:12:17.698555 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:12:17.700711 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.702137 10583 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
W20260709 12:12:17.660043 16624 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:12:17.572743 16585 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.709928 16585 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.568265 16561 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.617336 16626 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:17.666411 16608 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.667235 11217 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:12:17.721105 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87258.4us
I20260709 12:12:17.655850 16613 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.679086 16482 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:17.600447 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.685408 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.690659 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:17.615157 16469 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.722769 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.729729 11215 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:17.745890 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.746050 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.753451 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:17.710592 16373 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:17.757308 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.749497 10429 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:17.753885 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:17.717125 16451 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:17.723829 16604 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:17.765507 16627 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 12:12:17.766124 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:17.766778 16628 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:12:17.767637 16630 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 12:12:17.768340 16623 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 12:12:17.797369 16563 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.762800 16617 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:17.813282 11220 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:17.759513 16633 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.177s	user 0.002s	sys 0.000s
I20260709 12:12:17.772082 16422 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:17.844157 16600 consensus_meta.cc:287] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
I20260709 12:12:17.844796 16600 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:17.772794 16561 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.721617 16541 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:17.846959 16641 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.847824 16641 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.852705 10508 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:17.852878 11220 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:17.760602 16626 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:17.713471 16093 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:12:17.786211 16608 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.791293 16616 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:17.801501 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.752139 16569 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:17.806458 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 247594us
I20260709 12:12:17.768986 16566 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:17.816363 16562 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.826185 10552 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:17.841022 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:17.967886 10509 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:17.754846 16482 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.857982 16615 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:17.971860 16615 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:17.975525 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 171758us
I20260709 12:12:17.978230 16482 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.987190 16581 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:17.991386 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 165315us
I20260709 12:12:17.992656 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:17.993870 16587 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:17.994395 16587 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:17.995409 16587 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.005138 16469 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.006948 10437 raft_consensus.cc:2368] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
I20260709 12:12:17.811455 10438 raft_consensus.cc:2955] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index 7 to 10, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:17.946650 10336 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 16, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:17.919180 16562 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:17.875324 16628 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:12:17.898523 16535 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.878476 16641 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:17.885499 16632 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.293s	user 0.001s	sys 0.001s
I20260709 12:12:17.887003 16617 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:17.938750 16524 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:17.940511 16430 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:17.892722 16591 consensus_meta.cc:287] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.003s
I20260709 12:12:17.942624 16584 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:17.885171 16626 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:18.002702 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 124259us
I20260709 12:12:18.019179 16626 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.020275 16640 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:12:17.884470 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 83085.2us
I20260709 12:12:18.023015 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:18.024118 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:18.026602 16617 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:18.027037 16617 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.027478 16617 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:17.873826 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:17.831233 16563 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.031908 16584 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:18.052304 16625 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:18.054229 16625 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:18.057411 11178 leader_election.cc:336] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.834s (SENT)
I20260709 12:12:18.068575 16562 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.075389 16563 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:18.077201 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 203466us
I20260709 12:12:18.078141 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.079753 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:18.080761 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:18.084270 16625 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.096195 10587 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:12:17.942302 16635 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.278s	user 0.000s	sys 0.003s
I20260709 12:12:17.893638 16566 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:17.905431  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:18.103339 10510 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:12:18.086410 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.105569  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:18.092546 10336 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 14 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:18.106550  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:17.959295 16596 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.087574 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:18.110672 10511 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:18.113035 10509 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:12:18.087119 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47321.7us
I20260709 12:12:18.115584 10510 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:12:18.121451 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:18.123837 10585 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:18.126924 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:18.130038 11215 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.122470 16563 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:18.110512 16584 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.156172 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:18.158788 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:18.177711 10434 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:18.164003 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.180363 11220 raft_consensus.cc:2410] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:12:18.183015 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:18.183939 10437 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:18.188750 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.190776 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.195974 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.196802 10433 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:18.176925 10511 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
I20260709 12:12:18.143764 16591 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.164917 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.126102 16430 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:12:18.164064 10403 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.919s (SENT) [suppressed 41 similar messages]
I20260709 12:12:18.119671 16451 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:18.167328 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.168017 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.171829 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:18.179270 16383 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:18.155475 16563 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:18.163333 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:12:18.140851 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54546.1us
I20260709 12:12:18.214003 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:18.216756 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45066.9us
W20260709 12:12:18.203992 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89063.2us
I20260709 12:12:18.166285 16649 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.177899 16357 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:18.229529 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:18.227836 10430 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:18.228910 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:18.226845 11215 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:18.227550 10434 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:18.229522 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.236342 16357 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.220296 16353 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:12:18.252534 16357 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:18.237576 16649 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.253940 15699 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.254916 15699 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.256340 16649 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.259285 16572 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.259989 16572 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.239117 16430 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:18.264484 16430 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.226224 16093 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:12:18.258591 15699 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:18.285163 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.280450 11215 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:18.280901 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.280450 10437 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:12:18.286280 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57452.3us
I20260709 12:12:18.189354 16445 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:12:18.291929 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.292847 11213 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:18.229848 16563 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:18.302675 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75324.4us
I20260709 12:12:18.307689 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.310788 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.324286 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.302305 16581 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.325978 16373 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:18.326371 16581 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.217625 16591 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.330698 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:18.330793 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:18.306190 16353 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:18.338063 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.321156 16469 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.309476 16357 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:18.230137 16451 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.317307 16435 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:18.351493 11220 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:18.339709 16373 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:18.329630 16599 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:18.358388 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.358412 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:18.360987 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.362807 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.334000 16572 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:18.345088 16490 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:18.429809 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71123.7us
I20260709 12:12:18.368119 16568 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:18.438464 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 274429us
I20260709 12:12:18.369560 16599 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:18.443959 16599 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.445982 16451 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:18.451844 16562 raft_consensus.cc:993] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:18.458848 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.459537 11212 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:18.461862 16451 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:12:18.464190 16656 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:12:18.466125 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:18.473192 16591 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:18.475921 16649 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.478219 11213 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:18.477624 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 19 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:18.437285 16560 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:18.479952 16649 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.424934 16566 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.439076 16568 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:18.484131 16568 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.484714 16581 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:18.487799 16655 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:12:18.379505 16357 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:18.493549 16649 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:18.470865 10432 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:18.494822 10336 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 17, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 17 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:18.495992 11213 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:18.503717 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:18.510432 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:18.516885 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:18.522732 16626 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:18.524820 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:18.451874 16650 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:12:18.533164 11214 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 771) took 1044 ms. Trace:
I20260709 12:12:18.457710 16599 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.437852 16520 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:18.537930 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104895us
I20260709 12:12:18.537194 11214 rpcz_store.cc:276] 0709 12:12:17.488376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.488612 (+   236us) service_pool.cc:224] Handling call
0709 12:12:17.494374 (+  5762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:17.711296 (+216922us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:18.101796 (+390500us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.427536 (+325740us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.428937 (+  1401us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:18.429002 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:18.429053 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:18.471779 (+ 42726us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b18003366a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:18.493289 (+ 21510us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b18003366a0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:18.532870 (+ 39581us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540052f3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:18.532938 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:18.533092 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90361,"prepare.queue_time_us":255351,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":105,"raft.queue_time_us":39515,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":10725,"spinlock_wait_cycles":580771328,"thread_start_us":673737,"threads_started":2,"wal-append.queue_time_us":389111}
I20260709 12:12:18.470793 16596 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.542111 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.551715 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:18.552277 11213 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
W20260709 12:12:18.543648 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 202195us
I20260709 12:12:18.547443 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:18.512110 16648 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:12:18.376657 16373 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.387156 16587 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.134s
I20260709 12:12:18.555639 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.559321 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:18.729077 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 169859us
W20260709 12:12:18.716307 16650 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 12:12:18.731055 16660 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:12:18.584357 16613 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 12:12:18.733085 10435 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2509) took 1166 ms. Trace:
I20260709 12:12:18.733862 16613 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:18.735181 16599 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:18.736352 16617 consensus_meta.cc:287] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.708s	user 0.013s	sys 0.007s
I20260709 12:12:18.736838 16617 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.739841 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:18.562160 16520 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.740633 10582 raft_consensus.cc:3060] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:18.742503 16520 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.747726 16469 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:18.733522 10435 rpcz_store.cc:276] 0709 12:12:17.566748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.849129 (+282381us) service_pool.cc:224] Handling call
0709 12:12:17.849573 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:17.869489 (+ 19916us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:17.885878 (+ 16389us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.223167 (+337289us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:18.223802 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:12:18.223946 (+   144us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:18.349030 (+125084us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.362280 (+ 13250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:18.362363 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:18.732818 (+370455us) raft_consensus.cc:1685] finished
0709 12:12:18.732846 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:18.733013 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87635,"prepare.queue_time_us":245896,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":46551,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":9353,"thread_start_us":405733,"threads_started":2,"wal-append.queue_time_us":446900}
I20260709 12:12:18.554281 16626 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:18.751843 16617 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:18.753908 10582 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:18.756103 16560 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:18.756601 16560 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:18.758560 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.759183 16666 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.759745 16666 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:18.760991 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.761765 10586 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:18.761991 16666 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:18.763471 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:18.766373 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 211080us
I20260709 12:12:18.768061 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:18.769845 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.770325 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:18.770925 10581 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:18.549119 16615 consensus_meta.cc:287] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.577s	user 0.015s	sys 0.000s
W20260709 12:12:18.726117 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 177453us
I20260709 12:12:18.748194 16469 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:18.526242 16649 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:18.774600 10585 raft_consensus.cc:2368] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16.
I20260709 12:12:18.770493 10580 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 9.
I20260709 12:12:18.778690 16469 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:18.779351 16490 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.565107 16616 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.201s
I20260709 12:12:18.790185 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:18.729287 16567 raft_consensus.cc:1081] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
W20260709 12:12:18.730089 16535 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.716s	user 0.014s	sys 0.015s
I20260709 12:12:18.804381 10553 leader_election.cc:400] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16.
W20260709 12:12:18.806977 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:18.803900 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.818683 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2506) took 1252 ms. Trace:
I20260709 12:12:18.820616 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.821303 11214 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:12:18.831437 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:18.832154 10512 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:18.832671 10581 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:18.833473 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:18.836004 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:18.836622 10512 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.784631 16599 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:18.845170 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:18.866847 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 307015us
I20260709 12:12:18.820904 16435 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:18.839779 16558 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:18.876998 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.877851 10435 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:18.793424 16490 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:18.852898 16520 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.887646 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.852618 16664 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.803601 16568 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.878887 16558 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:18.866225 16451 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.871658 16640 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.896507 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.897905 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.899224 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:18.899776 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:18.955900 10434 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.900498 10431 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.901878 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:18.902927 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:18.957222 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:18.914908 10506 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:18.957770 10511 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.883285 16666 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:18.969918 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:18.949771 10512 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:18.880671 16291 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.891788 16665 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:18.954430 16652 log.cc:927] Time spent T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:12:18.985112 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:18.953130 16558 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:18.954146 16093 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:18.969647 16674 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.972023 16451 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.003798 16674 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.005402 16451 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:18.997807 16667 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:12:19.006280 16674 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:18.982511 16666 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.008127 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.000661 16626 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:19.008842 10580 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:19.009145 16666 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:19.009024 16626 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.009804 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.972532 16640 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.008054 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:18.974205 16616 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.916898 16664 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.022876 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.010926 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:18.912165 16520 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.988195 16581 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:19.033440 10434 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
W20260709 12:12:19.013659 16671 log.cc:927] Time spent T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:12:19.024457 10512 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:19.035712 10578 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:19.012650 16373 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.057056 10507 raft_consensus.cc:1240] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:19.011325 16626 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.023013 16674 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:18.994771 16541 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 12:12:19.053272 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 219923us
W20260709 12:12:19.054761 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79680.6us
I20260709 12:12:19.011965 16640 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:19.069020 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.053768 11215 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:19.059975 11213 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:19.061152 10552 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:19.071761 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:19.051020 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 222384us
I20260709 12:12:18.989928 16625 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:12:19.073117 10478 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:19.073447 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:19.013068 16093 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:19.056404 16524 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 1.040s	user 0.014s	sys 0.009s
W20260709 12:12:19.060714 16383 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.826s	user 0.017s	sys 0.009s
I20260709 12:12:19.075730 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:19.077016 16626 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:12:19.081300 16469 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.047720 16581 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.027320 16616 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:18.989629 16665 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.082336 16520 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.070226 16674 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.083595 16568 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.083664 16587 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:19.031517 16560 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.083617 16291 raft_consensus.cc:2804] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:19.089864 16599 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:12:19.095187 16676 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:12:19.099179 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:19.100838 10504 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:19.104724 10511 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:19.154294 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.102938 16435 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:19.156952 10581 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:18.819161 10429 rpcz_store.cc:276] 0709 12:12:17.565972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.768010 (+202038us) service_pool.cc:224] Handling call
0709 12:12:17.768502 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:17.804047 (+ 35545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:17.804091 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:17.815244 (+ 11153us) log.cc:844] Serialized 52 byte log entry
0709 12:12:18.307263 (+492019us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:18.793310 (+486047us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.818111 (+ 24801us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.818328 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:18.818403 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:18.818469 (+    66us) raft_consensus.cc:1685] finished
0709 12:12:18.818488 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:18.818624 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":115198,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":17623,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":152,"spinlock_wait_cycles":323328,"thread_start_us":835798,"threads_started":2,"wal-append.queue_time_us":573499}
I20260709 12:12:19.104885 16093 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.105629 16383 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:19.107801 10404 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:19.160821 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53116.1us
I20260709 12:12:19.163270 16649 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:19.163653 16649 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.164003 16649 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:19.167418 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.109023 16490 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:19.168284 11215 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:19.113732 16469 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.115576 16581 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:19.160251 16679 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:12:19.117336 16616 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.118211 10435 raft_consensus.cc:2368] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:12:19.173043 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.124571 16430 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:19.173882 16430 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.126586 16674 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:19.175384 10504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2823) took 1696 ms (client timeout 2103 ms). Trace:
I20260709 12:12:19.127489 16587 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.176054 16587 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:19.128690 16563 consensus_meta.cc:287] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.806s	user 0.006s	sys 0.008s
I20260709 12:12:19.174297 16430 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.179529 16563 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.181890 16563 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.129199 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:19.167559 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:19.183149 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72158.7us
I20260709 12:12:19.183380 11218 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:19.184367 16431 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:19.184962 16431 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:19.186187 10508 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:19.185928 16431 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.129606 16560 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.130064 16291 raft_consensus.cc:697] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:19.188189 16680 log.cc:927] Time spent T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:12:19.188516 16291 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.132181 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:19.190901 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58828.7us
I20260709 12:12:19.132513 16694 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:19.192759 10478 leader_election.cc:400] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16.
W20260709 12:12:19.136286 16689 log.cc:927] Time spent T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:12:19.193089 10478 leader_election.cc:403] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:19.194208 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:19.139294 10432 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:12:19.195102 16524 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 15. Current term is 16.
W20260709 12:12:19.117321 16686 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 12:12:19.198051 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2832) took 1104 ms. Trace:
I20260709 12:12:19.198443 10508 rpcz_store.cc:276] 0709 12:12:18.093970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.094159 (+   189us) service_pool.cc:224] Handling call
0709 12:12:18.122953 (+ 28794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:19.186568 (+1063615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.197815 (+ 11247us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540035ff54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:19.197988 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444673920}
I20260709 12:12:19.191597 16482 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:19.107558 10552 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:19.175791 10504 rpcz_store.cc:269] 0709 12:12:17.478853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.479099 (+   246us) service_pool.cc:224] Handling call
0709 12:12:19.175205 (+1696106us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400197dd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:19.175320 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036641536}
I20260709 12:12:19.148490 10336 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 12, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 12 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:12:19.200779 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 93328.2us
I20260709 12:12:19.148119 10583 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:19.201467 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:19.202312 16569 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.105671 10505 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:12:19.202834 16569 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:19.204006 10505 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 2813) took 3455 ms (client timeout 1919 ms). Trace:
W20260709 12:12:19.204371 10505 rpcz_store.cc:269] 0709 12:12:15.748844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.752818 (+  3974us) service_pool.cc:224] Handling call
0709 12:12:19.203776 (+3450958us) spinlock_profiling.cc:243] Waited 3.35 s on lock 0x7b54002c6ad4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:12:19.203899 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7708307328}
I20260709 12:12:19.204939 16674 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:19.205231 16569 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:19.205410 16599 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:19.128466 16664 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.205859 16569 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:19.205484 16674 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.208455 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.209089 10435 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:19.206099 16599 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:19.210132 16674 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.206813 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.205228 16609 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.211169 16609 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.140858 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:19.212769 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 49890.8us
I20260709 12:12:19.215102 16609 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.206852 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.217147 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:55454
I20260709 12:12:19.207680 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.199860 16482 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:19.168156 16687 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:12:19.219101 16482 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:19.202175 10336 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:19.224291 10336 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 7 to 11, term changed from 13 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:19.226596 10508 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:19.210316 16599 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.231689 10404 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:19.244356 16469 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:19.244933 16469 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.245309 16469 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:19.246953 10330 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 237) took 40171 ms (client timeout 15000 ms). Trace:
I20260709 12:12:19.234465 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:19.232682 16665 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:19.257510 10434 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:19.258580 10437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:19.263638 10437 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:19.251794 11212 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:19.264020 10437 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.264441 16609 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:19.267349 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:19.268142 16683 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:19.268545 16683 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.268882 16683 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.271680 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:19.272087 16693 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:19.274091 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.274358 16693 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:19.274976 16693 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:19.276021 16693 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.277158 16693 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:19.287036 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:19.287982 16616 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:19.288511 10403 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:19.303948 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.304332 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.272728 10336 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 4 to 10, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:19.309904 10336 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:19.313683 10336 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 12, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:19.320935 16558 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.371347 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:19.372911 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:19.375306 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:19.323490 11215 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:19.330804 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.376709 10582 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:12:19.377645 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.378643 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.379565 10582 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:12:19.381127 10582 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:12:19.335392 10587 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:12:19.341187 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52767.7us
I20260709 12:12:19.350375 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.383901 10581 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:12:19.352916 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.354905 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:19.390303 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 82116.3us
I20260709 12:12:19.390780 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:19.402390 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.404350 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.355369 10336 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 16, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:19.406757 10336 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:19.391629 10404 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:19.345413 10437 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:19.334565 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 62966.6us
I20260709 12:12:19.407431 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:19.262674 16649 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.408596 10336 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:19.367110 16626 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:19.367430 16616 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:19.366791 16609 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:12:19.261766 10330 rpcz_store.cc:269] 0709 12:11:39.075755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.076279 (+   524us) service_pool.cc:224] Handling call
0709 12:12:19.246777 (+40170498us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:14.584367 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:14.585254 (+   887us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:14.589605 (+  4351us) write_op.cc:435] Acquired schema lock
0709 12:12:14.593012 (+  3407us) tablet.cc:662] Decoding operations
0709 12:12:14.651377 (+ 58365us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:14.677156 (+ 25779us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:14.691294 (+ 14138us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:14.752701 (+ 61407us) tablet.cc:701] Row locks acquired
0709 12:12:14.755557 (+  2856us) write_op.cc:260] PREPARE: finished
0709 12:12:14.773045 (+ 17488us) write_op.cc:270] Start()
0709 12:12:14.773194 (+   149us) write_op.cc:276] Timestamp: P: 1783599134772980 usec, L: 0
0709 12:12:14.773221 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:12:14.773946 (+   725us) log.cc:844] Serialized 535 byte log entry
0709 12:12:15.270740 (+496794us) op_driver.cc:464] REPLICATION: finished
0709 12:12:15.442328 (+171588us) write_op.cc:301] APPLY: starting
0709 12:12:15.449065 (+  6737us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:15.493151 (+ 44086us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:15.493266 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:15.494121 (+   855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:15.494331 (+   210us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:12:15.494384 (+    53us) write_op.cc:312] APPLY: finished
0709 12:12:15.494624 (+   240us) log.cc:844] Serialized 25 byte log entry
0709 12:12:15.494817 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:15.494916 (+    99us) write_op.cc:454] Released schema lock
0709 12:12:15.494999 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:15.495453 (+   454us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:15.495491 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":270477,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":136018,"apply.run_cpu_time_us":4067,"apply.run_wall_time_us":69716,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":141671,"prepare.run_cpu_time_us":52906,"prepare.run_wall_time_us":346251,"raft.queue_time_us":105520,"raft.run_cpu_time_us":3978,"raft.run_wall_time_us":116393,"replication_time_us":483324,"thread_start_us":359213,"threads_started":4,"wal-append.queue_time_us":268110,"wal-append.run_cpu_time_us":11479,"wal-append.run_wall_time_us":1474267}]]}
I20260709 12:12:19.440076 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.440771 11215 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:12:19.432344 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.390009 16674 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:19.437567 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.442914 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.444905 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.445062 10437 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.415966 16581 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:19.434252 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:19.403234 16691 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:19.411715 16541 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:19.405179 16617 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:19.464879 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:19.445708 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 44772.6us
I20260709 12:12:19.436316 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:19.420040 16535 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:19.489804 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53575.8us
I20260709 12:12:19.490531 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:12:19.479929 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45791.9us
I20260709 12:12:19.414688 16677 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:12:19.482388 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 96257.8us
I20260709 12:12:19.400904 16587 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.493029 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.496670 16435 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:19.434959 16695 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:19.504340 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:19.418383 16664 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:12:19.518214 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:19.519302 16695 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:19.479203 16524 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:19.524473 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:19.543677 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.545255 10434 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:19.545918 10509 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:19.548031 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:19.483217 16563 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:19.508472 16520 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:19.554633 16520 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.488487 16691 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.544714 16695 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:19.555109 16520 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:19.500203 16535 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.516393 16599 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:19.508621 16604 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:19.470006 16561 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:12:19.475852 16674 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.541810 16445 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:19.568347 16357 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:19.481980 16581 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.578814 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.531561 16435 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.473637 16430 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.576428 16649 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.563874 16572 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:12:19.602823 16626 consensus_queue.cc:1059] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:19.635694 16649 raft_consensus.cc:3055] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:19.636124 16649 raft_consensus.cc:740] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:19.637007 16649 consensus_queue.cc:260] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.574450 16482 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.616710 16587 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:19.617542 16435 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:19.649044 16482 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.623674 10508 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18.
I20260709 12:12:19.624670 16430 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.516491 16617 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.652094 16469 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.654002 16617 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.565004 16535 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.505155 16541 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.660674 16541 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.627213 16581 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.627756 10579 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:12:19.629222 16691 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.632395 16683 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.633488 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 0b474d9b241d4a698702287ee1bbe3b2 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:19.666455 16683 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.569142 16604 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.670496 16604 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.643313 16445 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:12:19.612529 16699 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:12:19.617729 16558 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.558096 16567 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 12:12:19.675411 16445 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:19.676369 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66670.7us
I20260709 12:12:19.679095 16520 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.569491 16625 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 12:12:19.708596 16625 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:19.709268 16625 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.710297 16674 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.737330 10584 raft_consensus.cc:2410] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:19.756979 11220 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:19.758234 16535 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.760717 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.750609 16674 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.738354 16616 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:19.764024 10511 raft_consensus.cc:2368] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18.
I20260709 12:12:19.766063 16535 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.758872 16469 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.737154 16695 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:19.783910 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 259160us
W20260709 12:12:19.733174 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 196756us
I20260709 12:12:19.775703 16625 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.785056 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:19.774313 16674 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.777423 16616 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.804421 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:19.813937 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 295697us
I20260709 12:12:19.796535 16353 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:19.827594 16674 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.819535 16721 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:19.829310 16721 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:19.842133 16445 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.846691 16469 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:19.847126 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.801024 16677 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:19.848044 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 326351us
I20260709 12:12:19.852617 16616 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.830597 16721 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.861697 16677 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.863454 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:19.864528 16572 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:19.866181 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:19.867274 16520 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.868062 16584 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:19.865039 16641 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:19.868336 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.869527 16695 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:19.870539 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:19.871440 16535 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:19.871857 16535 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.872205 16535 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:19.873111 16445 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:19.848125 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43868.9us
I20260709 12:12:19.876471 16674 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:19.877400 16674 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:19.877715 16674 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:19.880698 16562 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:19.881145 16562 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:19.882015 16562 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.882861 16535 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:19.884385 16524 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:19.884922 16524 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:19.885197 16535 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.892468 10578 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:12:19.823105 16718 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:19.895299 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.895519 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.895955 10578 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:19.896117 10586 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:12:19.878612 16674 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.897603 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.898089 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:19.901217 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:19.901851 10432 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.903523 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.904429 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:19.905328 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.905520 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:19.905913 10586 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.909019 10508 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:12:19.909544 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.911142 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.911906 10583 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:19.913007 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.913763 11218 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.848192 16353 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.915864 16353 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:19.913101 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:19.919353 10509 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:19.921272 10552 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18.
I20260709 12:12:19.898752 16674 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:19.925155 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:19.942557 10429 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.926354 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:19.921531 10586 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:19.921711 10552 leader_election.cc:403] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:19.933084 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:19.944952 16445 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.948247 10584 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:19.935976 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:19.924844 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.934681 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:19.951910 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.952684 10581 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:19.952867 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:19.929286 11218 raft_consensus.cc:2410] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:12:19.920096 16562 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:19.943663 10434 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:19.960740 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:20.056133 11214 raft_consensus.cc:2368] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:12:19.961643 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.962714 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.057170 10578 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:19.922760 16718 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:19.922917 16520 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:20.047722 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:19.929960 16711 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:20.053136 10432 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:20.060103 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.061867 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.062376 10579 raft_consensus.cc:2393] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 15: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:19.923070 16649 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:19.964783 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.065090 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.064986 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:20.070578 11214 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:20.078532 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.077014 10429 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:20.079161 10434 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:20.091012 10507 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:20.104933 10553 leader_election.cc:400] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:12:20.106508 10553 leader_election.cc:400] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:12:20.108184 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.109163 10435 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:12:20.111431 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.113538 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.115610 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.117779 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.119874 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.120754 10437 raft_consensus.cc:2368] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18.
I20260709 12:12:20.121899 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.124943 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.106822 10553 leader_election.cc:403] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:12:20.129582 16604 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.059113 16711 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.081652 16725 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:20.132535 16725 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.133718 16711 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:20.133934 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.136282 11213 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 15: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:12:20.138521 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.140440 10507 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:12:20.142200 10434 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:20.143405 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.143608 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.145222 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.126088 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.151284 10437 raft_consensus.cc:2368] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14.
I20260709 12:12:20.152865 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.145222 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.154948 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.155959 10435 raft_consensus.cc:2368] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:12:20.145469 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.154374 10431 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:20.156764 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.164232 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.167171 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.154139 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.179267 16563 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:20.158586 16357 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:12:20.187417 11215 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:20.158912 16642 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:20.189071 16725 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:20.189294 16563 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:20.189733 16642 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.193028 16718 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:20.193418 10431 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:20.194399 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.195747 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.196933 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.149031 16693 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 12. Current term is 13.
I20260709 12:12:20.197489 16642 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:20.199565 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:20.159502 16640 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.220444 16640 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.223227 16604 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:20.113587 16524 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.208148 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.210925 10579 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:20.210845 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.214651 16445 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18.
I20260709 12:12:20.226490 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:20.202575 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.232419 10431 raft_consensus.cc:1813] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Handling vote request from an unknown peer 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.233260 10431 raft_consensus.cc:2410] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:12:20.227900 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73866.3us
I20260709 12:12:20.230507 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.235044 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.237351 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.252988 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.253762 10583 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:20.254437 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.229818 16723 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.261751 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.235594 16574 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:20.264977 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:20.239194 16535 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:20.236824 16722 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:12:20.230204 16567 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:12:20.271811 16625 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:20.304327 10479 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:20.279201 16572 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:20.287662 10583 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:20.330286 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.289285 16524 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:20.290007 16469 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:20.244105 16725 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:12:20.292243 10580 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:20.262061 16721 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:20.299885 10506 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:20.338006 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.245028 16558 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:12:20.306128 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 64105.7us
I20260709 12:12:20.275825 16383 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:20.335601 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:20.299417 16726 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:20.349112 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.337917 16616 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:20.330466 16640 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:20.326565 16695 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:20.328994 16626 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.354851 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:20.359006 10582 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:20.361424 10583 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:20.359933 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 54046.4us
I20260709 12:12:20.371774 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:20.361351 10579 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:20.373767 16535 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:20.341576 16566 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:20.379347 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.345970 16383 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:20.352049 16616 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:20.370793 16299 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:20.383208 16469 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:20.397763 16469 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:20.398211 16469 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:20.400120 16616 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.401549 16616 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:20.377739 16726 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:20.404917 16726 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.375574 16695 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.417770 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:20.422039 10506 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:20.428673 16722 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:20.429386 16722 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.431421 10505 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:12:20.435149  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:20.456279 10583 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 11.
I20260709 12:12:20.420112 16626 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:20.464800 10587 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:20.451486 10433 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 7 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:20.459270 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:20.477656 10403 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:20.456872 16725 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:20.460421 16566 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:20.504792 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 114525us
I20260709 12:12:20.456959 16383 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.509435 16566 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.512861 16674 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:20.513612 16674 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:20.514804 16482 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:20.515707 16482 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.518646 16482 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:20.547510 16722 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:20.485484 16721 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:20.490724 16574 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:12:20.500233 16725 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:20.557289 16725 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.462713 16626 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.561424 16626 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:20.562143 16357 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.562814 16357 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.565747 16357 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:20.506878 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:20.584654 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77879.2us
I20260709 12:12:20.587323 11177 leader_election.cc:400] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18.
I20260709 12:12:20.587723 11177 leader_election.cc:403] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:20.587370 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.597841 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.499274 16299 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:20.604198 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 104314us
I20260709 12:12:20.595168 16535 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.592327 16520 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:20.620074 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.599026 16718 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:20.628849 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:12:20.629401 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.630084 10434 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:20.626452 10435 raft_consensus.cc:2393] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:20.633680 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:20.657415 10431 raft_consensus.cc:2393] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:20.658510 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.514786 16674 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.690002 10586 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:20.690030 10553 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:12:20.692365 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:20.693099 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.694766 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.696058 16357 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:20.696952 16357 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:20.698704 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.699761 16722 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.704350 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.709229 10553 leader_election.cc:400] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18.
I20260709 12:12:20.710760 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.712785 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.714946 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:20.712779 16469 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.710582 16357 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:20.735774 16740 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:12:20.735780 16469 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:20.682974 16299 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.758816 10404 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:20.760407 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:20.767504 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:20.792723 10579 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:12:20.769145 16581 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:20.795089 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:20.773813 16742 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 12:12:20.815299 16677 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:20.816133 16677 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.781685 16563 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:20.800863 16585 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:20.805652 16626 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.821614 16711 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.824455 16677 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:20.835819 16585 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.828450 16563 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:20.871680 16585 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:20.874168 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 163362us
I20260709 12:12:20.872926 16563 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:20.887537 11213 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:20.889060 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:20.889568 11178 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.891047 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.893045 10553 leader_election.cc:400] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14.
I20260709 12:12:20.893446 10553 leader_election.cc:403] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:20.894245 10582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:20.895087 10582 raft_consensus.cc:606] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.849116 16711 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:20.909327 11220 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:20.914083 16569 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.949501 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:20.955544 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62701.9us
I20260709 12:12:20.885162 16585 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:20.958773 11213 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:12:20.965777 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.972008 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.911283 16581 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:12:20.901885 16722 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:20.972872 10478 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.973536 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.973912 10553 leader_election.cc:400] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:12:20.981244 10553 leader_election.cc:403] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:12:20.960970 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.975033 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:20.976626 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:20.986142 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.938725 16520 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18.
W20260709 12:12:20.996243 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106863us
I20260709 12:12:20.972746 16677 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:20.998057 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:20.977967 16625 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:20.996392 16721 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:12:20.988946 16566 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:21.007817 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:20.982801 16469 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:12:20.977742 16357 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:12:20.988690 16626 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:20.975651 16585 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.007243 16616 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:21.043222 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45256.5us
I20260709 12:12:21.044520 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:21.001343 16725 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:21.041620 16585 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:21.020658 16524 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.010033 16693 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:12:21.009580 16563 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:12:21.062748 16752 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:12:21.008865 16535 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 12:12:21.068652 16749 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.170s	user 0.000s	sys 0.004s
W20260709 12:12:21.069124 16747 log.cc:927] Time spent T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:12:21.072013 16695 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:21.079562 16750 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:12:21.048611 16625 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:21.058032 16616 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.058593 16674 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:21.085759 16616 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:21.023171 16357 consensus_queue.cc:1243] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:12:21.087976 16711 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.088560 16760 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:21.089756 11215 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:21.089434 16563 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.093317 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:21.079882 16743 log.cc:927] Time spent T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.414s	user 0.002s	sys 0.000s
I20260709 12:12:21.098095 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:21.100886 11215 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
W20260709 12:12:21.102029 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 94306.9us
I20260709 12:12:21.048861 16566 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:21.111125 10433 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:21.107421 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:21.116356 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:21.120256 11215 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 21: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:12:21.094435 16524 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:21.129436 16753 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:12:21.099745 16560 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:12:21.119510 16725 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:21.101414 16711 consensus_queue.cc:1059] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:12:21.178399 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.179555 10507 raft_consensus.cc:2393] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:21.179849 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.183072 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:21.194236 16581 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 13. Current term is 14.
I20260709 12:12:21.172278 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.173708 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.216293 10513 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:12:21.177079 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.194864 16524 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.195880 16563 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:21.196241 10508 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:21.199355 16299 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:21.171080 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 197204us
I20260709 12:12:21.203194 16383 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:21.207314 16469 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:21.177887 16725 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.226512 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:21.240568 10584 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:21.271219 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:21.272411 10580 raft_consensus.cc:2410] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:21.286978 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:21.287824 11213 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 12:12:21.288389 11213 pending_rounds.cc:85] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 11
I20260709 12:12:21.288621 11213 pending_rounds.cc:107] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:12:21.288847 11213 raft_consensus.cc:2889] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:21.155246 16566 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.268573 16725 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.233414 16482 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:12:21.266650 16469 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:21.304893 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.305910 10432 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
W20260709 12:12:21.313858 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 269469us
I20260709 12:12:21.317054 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.317234 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.317503 11178 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:21.318576 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.320168 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:21.322558 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:21.327421 10578 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:12:21.331789 10585 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:12:21.332243 10585 pending_rounds.cc:85] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 8
I20260709 12:12:21.332463 10585 pending_rounds.cc:107] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:12:21.332693 10585 raft_consensus.cc:2889] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:21.335253 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.336184 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.336830 10504 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:21.339279 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.343323 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.344846 10504 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:21.337738 16520 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:21.359460 16469 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.372278 16469 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:21.372378 16572 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:21.375185 16572 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:21.376315 16572 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.378058 16572 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:21.365512 11221 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:12:21.381858 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:21.386561 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 106003us
I20260709 12:12:21.389037 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.364682 16677 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:21.367460 16626 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:21.390520 16677 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:21.392555 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 149723us
W20260709 12:12:21.393640 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.393805 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:21.406253 16677 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.407722 16677 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:21.415917 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:21.431308 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.432116 10431 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:12:21.390849 16626 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.409979 16766 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:21.457960 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:21.425305 16760 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.407734 16761 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.424317 16625 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
W20260709 12:12:21.451783 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 172758us
I20260709 12:12:21.410006 16765 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:12:21.480247 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:21.495194 16568 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:21.473786 16760 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.495680 16568 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:21.496158 16568 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:21.497545 16760 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.424894 16716 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:12:21.453907 16693 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:12:21.500433 16693 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:21.500850 16693 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:21.501871 16693 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.411538 16711 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:12:21.411790 16616 consensus_queue.cc:1059] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:12:21.489411 16766 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:21.483075 16761 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.518319 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.506263 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:21.486459 16626 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:21.509596 16751 consensus_queue.cc:1175] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" responder_term: 20 status { last_received { term: 20 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 20 index: 14 } }
I20260709 12:12:21.520586 16693 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:21.521621 16685 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.550837 16685 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.555642 16685 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:21.557463 16761 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:21.559131 16568 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.559851 16725 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.561476 16568 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:21.561733 16725 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:21.607645 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 181606us
W20260709 12:12:21.607384 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93670.7us
W20260709 12:12:21.613356 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 122323us
I20260709 12:12:21.639714 10430 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:21.656538 10432 raft_consensus.cc:2368] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17.
I20260709 12:12:21.645593 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:21.659436 11220 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.669587 10582 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:12:21.670573 16560 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.671800 16560 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.622920 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:21.681075 10510 raft_consensus.cc:1240] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 10. Current term is 11. Ops: []
W20260709 12:12:21.691622 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 77262.4us
I20260709 12:12:21.715739 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.717222 10513 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:21.718562 16641 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:21.719455 10435 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:21.727626 16725 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.728794 16725 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.710542 16572 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:21.732028 16674 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:12:21.731777 16572 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.732543 16674 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:21.734279 16674 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.738056 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:21.738502 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:21.739403 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:21.740594 16718 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:21.741257 16718 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.741720 16718 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.742436 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:21.751320 16445 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:12:21.754086 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 58185.3us
I20260709 12:12:21.758671 16642 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:21.763872 16560 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.768267 16674 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:21.777969 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.779286 16677 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:21.797238 10513 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:21.797356 16677 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:21.802878 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 60688.9us
I20260709 12:12:21.804823 16677 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.788353 16725 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.735072 16572 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:21.809237 16716 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:21.809996 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:21.792256 10432 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:21.796141 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:21.827273 16445 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:21.827867 16445 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.828284 16445 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:21.833410 10582 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:12:21.838662 11220 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:21.844141 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:21.866165 16353 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:21.866822 16353 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.867309 16353 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:21.872623 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:21.896452 16787 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:21.897454 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 113627us
I20260709 12:12:21.897531 16787 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:21.899299 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:21.901219 16780 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:21.903545 16787 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:21.901397 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:21.912410 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.917915 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:21.918429 10404 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:21.918855 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57531.3us
I20260709 12:12:21.919517 16780 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:21.921371 16780 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:12:21.911135 16793 log.cc:927] Time spent T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 12:12:21.922598 16780 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.920246 16787 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:21.931651 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:21.933352 16751 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.934214 16751 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.925917 16787 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.947423 16787 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.947463 10508 raft_consensus.cc:1240] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:21.949579 16482 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:21.955133 16751 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:21.917414 16674 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:21.956784 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:21.967026 16718 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:21.973116 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:21.976917 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 75705.5us
I20260709 12:12:21.978955 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:21.983238 16566 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:21.990643 10323 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet deab11eaf55b4f3ebe48926a1e2afa38 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 7 but the committed config has opid_index of 10
I20260709 12:12:22.003055 16718 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:22.004526 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:22.005324 10584 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.020542 16766 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:12:22.030455 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 113) took 8610 ms. Trace:
I20260709 12:12:22.035939 16445 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.061384 16445 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:22.039510 10584 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:12:22.049172 16566 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:22.065191 16566 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.066224 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:22.068292 16560 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:22.068674 16566 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.068907 16560 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:22.070379 16572 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.053261 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:22.072252 10508 raft_consensus.cc:606] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.049636 16783 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:22.070214 16560 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.059226 10336 rpcz_store.cc:276] 0709 12:12:13.419527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.419874 (+   347us) service_pool.cc:224] Handling call
0709 12:12:22.030095 (+8610221us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:19.859102 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:19.859804 (+   702us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:19.859850 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:12:19.859871 (+    21us) tablet.cc:662] Decoding operations
0709 12:12:19.861144 (+  1273us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:19.861214 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:19.861263 (+    49us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:12:19.862442 (+  1179us) tablet.cc:701] Row locks acquired
0709 12:12:19.862591 (+   149us) write_op.cc:260] PREPARE: finished
0709 12:12:19.908492 (+ 45901us) write_op.cc:270] Start()
0709 12:12:19.908675 (+   183us) write_op.cc:276] Timestamp: P: 1783599139908406 usec, L: 0
0709 12:12:19.908701 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:12:20.082495 (+173794us) log.cc:844] Serialized 5177 byte log entry
0709 12:12:20.768112 (+685617us) op_driver.cc:464] REPLICATION: finished
0709 12:12:20.942649 (+174537us) write_op.cc:301] APPLY: starting
0709 12:12:20.942760 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:21.099296 (+156536us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:21.117399 (+ 18103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:21.480948 (+363549us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:21.481240 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:12:21.481293 (+    53us) write_op.cc:312] APPLY: finished
0709 12:12:21.481679 (+   386us) log.cc:844] Serialized 121 byte log entry
0709 12:12:21.481888 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:21.482085 (+   197us) write_op.cc:454] Released schema lock
0709 12:12:21.482150 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:21.482665 (+   515us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:21.482699 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1397651,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":174218,"apply.run_cpu_time_us":25914,"apply.run_wall_time_us":539996,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58493,"num_ops":13,"prepare.queue_time_us":303601,"prepare.run_cpu_time_us":55735,"prepare.run_wall_time_us":501462,"raft.queue_time_us":340970,"raft.run_cpu_time_us":17962,"raft.run_wall_time_us":109799,"replication_time_us":859196,"thread_start_us":799123,"threads_started":4,"wal-append.queue_time_us":301741}]]}
W20260709 12:12:22.091333 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 112540us
I20260709 12:12:22.086248 16642 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:22.094621 10585 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:22.083066 16572 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.096511 16716 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:22.097489 16572 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.099014 16353 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.110646 16353 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:22.106833 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:22.116159 10585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:22.122428 10585 raft_consensus.cc:606] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.122867 10585 raft_consensus.cc:614] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:22.134685 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:22.135592 10580 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:22.128819 16783 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:22.121786 16787 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.153733 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:22.154735 10508 raft_consensus.cc:606] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.141258 16780 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:22.178102 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:22.178807 10579 raft_consensus.cc:606] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.179447 10579 raft_consensus.cc:614] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:22.186265  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:22.195358  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:22.196933  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:22.198307 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 103543us
I20260709 12:12:22.190205 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.195982 10332 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 8 to 9, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 9 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:22.208454 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 80784us
I20260709 12:12:22.208997 16566 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:22.189990 16780 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.232632 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.242722 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet e4cd5d330f694dacb199bd61266c8f5a with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51568
I20260709 12:12:22.251452 10437 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:22.241433 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:22.266279 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.287158 10580 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
I20260709 12:12:22.289676 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:12:22.290100 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 61248.1us
I20260709 12:12:22.296819 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:22.299070 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:22.299330 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:22.299889 10508 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.298108 10585 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:22.302919 10435 raft_consensus.cc:2410] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:12:22.287919 16445 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:22.299775 16718 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:22.299176 16783 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:22.305203 16787 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:22.325795 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.315948 16642 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.325794 16445 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.341591 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:22.320757 16716 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.349803 16801 tablet_replica.cc:333] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:12:22.350499 16520 consensus_queue.cc:1059] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 12:12:22.351583 16469 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:22.354122 16520 raft_consensus.cc:3055] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:22.354632 16520 raft_consensus.cc:740] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:22.355588 16716 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.355948 16520 consensus_queue.cc:260] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.357088 16780 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.357532 16520 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:22.360141 16716 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:22.360435 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:22.360900 16642 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.351509 16801 raft_consensus.cc:2243] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 12:12:22.386837 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 64706us
I20260709 12:12:22.387686 11214 raft_consensus.cc:2410] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:12:22.388806 10508 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:12:22.364430 16718 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:22.398275 16566 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.399145 16566 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:22.418663 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104111us
I20260709 12:12:22.412901 16642 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.432271 10513 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:12:22.462965 16801 raft_consensus.cc:2272] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:22.462800 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.464123 10437 raft_consensus.cc:2410] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:12:22.488222 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.489163 10580 raft_consensus.cc:2368] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17.
I20260709 12:12:22.490082 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.472328 16445 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:22.496450 10437 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:12:22.462018 16718 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.506436 16566 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:22.508008 16798 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.222s	user 0.005s	sys 0.000s
W20260709 12:12:22.508855 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.511197 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:22.512310 11212 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:22.516100 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.520010 16766 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:22.523465 10580 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:22.525039 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.525907 10584 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:22.531468 11220 raft_consensus.cc:2410] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:12:22.548683 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.549856 10432 raft_consensus.cc:2410] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:22.550225 16766 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.550748 16766 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.554059 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:22.554708 10508 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:22.556295 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:22.556967 10504 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.558250 16766 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:22.559504 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77329.3us
I20260709 12:12:22.553794 16716 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:12:22.564263 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.576076 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:22.574632 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.582269 16572 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:22.582543 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:22.583441 10510 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:22.590332 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.593520 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:22.596101 10580 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:12:22.596052 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.597178 10582 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:22.600145 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.573962 16445 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:22.603999 10579 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:12:22.605343 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:22.606016 10513 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:22.598138 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:22.607676 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:22.608352 10510 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.582789 16572 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:22.615352 16572 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:22.615651 16801 ts_tablet_manager.cc:1916] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:22.657200 10508 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:22.680070 10437 pending_rounds.cc:85] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 5
I20260709 12:12:22.680487 10437 pending_rounds.cc:107] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:12:22.680903 10437 raft_consensus.cc:2889] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:22.681335 10437 pending_rounds.cc:107] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:12:22.681630 10437 raft_consensus.cc:2889] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:22.682497 10504 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:12:22.679833 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 112173us
I20260709 12:12:22.683490 16783 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.691599 16783 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.694196 16783 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.695487 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:22.696530 16811 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:22.696988 16811 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:22.697394 16811 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:22.699905 10505 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:12:22.703352 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:22.704028 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:22.704802 11214 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:22.704874 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:22.707199 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:22.707249 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 143201us
I20260709 12:12:22.708657 16445 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:22.709231 16445 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:22.704226 16751 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:22.695411 10506 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:12:22.716504 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.718078 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.718894 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.720844 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.714249 16811 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.723307 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.724206 10431 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:22.727792 16811 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:22.730054 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:22.732282 16751 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:22.733460 16751 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.740583 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:22.748759 10510 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
W20260709 12:12:22.750209 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 173918us
I20260709 12:12:22.754613 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.710264 16445 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.747822 16766 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:22.763790 16766 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:22.764213 16766 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:22.768056 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:22.769804 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:22.771988 16469 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:22.772874 16469 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:22.773255 16469 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:22.722697 16572 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.773622 16469 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:22.785161 16780 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:22.808517 10507 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:22.825754 16783 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:22.789182 16353 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.803773 16642 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:12:22.830310 10580 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:12:22.854185 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 279625us
I20260709 12:12:22.831370 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:22.833544 16572 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:22.859491 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 89747.3us
W20260709 12:12:22.860580 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 149988us
I20260709 12:12:22.841358 16780 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:22.873157 16766 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:22.884353 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 179537us
I20260709 12:12:22.891037 16642 raft_consensus.cc:993] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:22.891198 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:22.902258 16642 raft_consensus.cc:1081] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:22.905915 10508 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:22.905666 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:22.906785 16828 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:22.908003 16828 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.905764 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.909384 16819 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:22.909811 16818 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.910233 16819 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.910553 16818 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.913182 16819 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:22.914973 16800 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:22.915328 16818 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.915549 16800 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:22.916031 16800 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:22.917003 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:22.920204 16826 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:22.921084 16826 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:22.923473 16780 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.924978 16780 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.928999 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:22.863968 16783 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:22.934193 10504 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:22.908936 16822 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:22.953394 16800 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.955722 16828 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:22.957852 16783 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:22.874473 16353 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:22.961160 16353 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:22.876052 16718 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 12:12:22.985024 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 153798us
I20260709 12:12:23.005481 16766 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:23.006115 10430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:23.006944 10430 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:23.015705 10430 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:23.050904 16787 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:12:23.089663 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.098945 10430 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.101176 16469 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.101612 10430 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:23.106117 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:23.176843 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70820.2us
W20260709 12:12:23.078974 16810 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:12:23.179143 16818 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:23.179728 16818 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:23.180850 16818 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.121882 16520 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.183398 16520 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.134958 16787 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.189829 16787 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:23.147938 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:23.005573 16801 ts_tablet_manager.cc:1929] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.9
I20260709 12:12:23.192044 16801 log.cc:1199] T e4cd5d330f694dacb199bd61266c8f5a P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/e4cd5d330f694dacb199bd61266c8f5a
I20260709 12:12:23.194403 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet e4cd5d330f694dacb199bd61266c8f5a (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:12:23.175586 10553 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.527s (SENT) [suppressed 3 similar messages]
I20260709 12:12:23.204355 16783 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:23.205116 16783 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.208146 16827 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.208717 16827 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.211501 16827 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:23.217882 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.218814 10506 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:23.222424 16783 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:23.230263 16469 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:23.234092 16800 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:23.235219 16825 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:23.236214 16825 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.200876 16520 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:23.281692 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 376091us
W20260709 12:12:23.316058 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 454669us
I20260709 12:12:23.318037 16825 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:23.319713 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 383778us
I20260709 12:12:23.327015 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:23.327809 10585 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:12:23.349499 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:23.351084 10580 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:12:23.357949 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:23.358690 11214 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:23.365532 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:12:23.367097 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 191564us
I20260709 12:12:23.375634 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:23.377314 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:23.377974 11213 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:23.378705 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:23.372874 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:23.384639 10505 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:23.315958 16469 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:12:23.389717 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.391702 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.397594 10510 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:23.398023 10508 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:23.393777 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.395679 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:23.395982 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.396347 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:23.386566 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.356930 16520 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:12:23.373589 16718 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:23.410905 10579 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:23.416051 10504 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:23.416626 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:23.417347 10506 raft_consensus.cc:2393] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:23.417276 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:23.415410 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:23.438876 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:23.428632 10585 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:23.439872 11221 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
W20260709 12:12:23.444453 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.448035 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:23.448987 10508 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:23.448105 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:23.458146 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:23.458925 10511 raft_consensus.cc:3055] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:23.459250 10511 raft_consensus.cc:740] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:23.460206 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:23.461105 10580 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.415032 16718 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.470602 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.473099 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:23.473830 10510 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.476400 10478 leader_election.cc:400] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17.
I20260709 12:12:23.476801 10478 leader_election.cc:403] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:23.476438 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:23.477730 10506 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:23.485720 10438 raft_consensus.cc:1240] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:23.487443 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:23.488683 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:23.495793 10510 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:23.495908 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:23.506156 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62434.3us
I20260709 12:12:23.428292 16800 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:12:23.510505 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:23.520673 10505 raft_consensus.cc:1275] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:12:23.471299 16718 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:23.508424 10553 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:23.477270 10511 consensus_queue.cc:260] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.575431 10511 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:23.576818 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 68545us
I20260709 12:12:23.548494 10435 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:12:23.511397 16780 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:23.580837 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:23.583740 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 87964.7us
I20260709 12:12:23.584689 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:23.585359 16816 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:23.591439 11214 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:23.593513 11213 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
W20260709 12:12:23.593698 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 128059us
I20260709 12:12:23.594970 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:23.602280 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 127431us
W20260709 12:12:23.602488 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 137991us
I20260709 12:12:23.599438 16828 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:23.606269 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:23.608933 10404 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:23.622961 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:23.625638 10505 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:23.631791 10513 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:12:23.636523 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:23.603574 16642 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17.
I20260709 12:12:23.638835 16469 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:23.639274 16839 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:12:23.639663 16445 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:23.639849 16469 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:23.640384 16469 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.640765 16469 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:23.605340 16829 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:12:23.642369 16829 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:23.643427 16800 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:23.643985 16829 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:23.644258 16800 consensus_queue.cc:1048] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:12:23.647986 16780 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:23.652334 10511 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:23.644464 16829 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:23.663229 10438 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:12:23.656390 10508 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:23.658216 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:23.664214 10404 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:23.664855 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:23.676609 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:23.678968 10403 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:23.670578 16828 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:23.676625 10580 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
W20260709 12:12:23.702925 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 80089us
I20260709 12:12:23.680172 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:23.706853 10404 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:23.707648 10436 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:23.708930 10478 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:23.711881 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:23.648464 16780 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:23.710150 16766 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:12:23.715451 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50672us
W20260709 12:12:23.717572 16833 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.358s	user 0.000s	sys 0.002s
W20260709 12:12:23.721938 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 127077us
I20260709 12:12:23.730436 16766 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:23.731057 16766 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:23.732072 16766 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.712531 16642 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:23.706640 16847 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:23.721052 16849 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:12:23.758069 16846 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:23.758893 16846 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:23.760066 16846 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.735536 16566 raft_consensus.cc:2764] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:12:23.763682 16804 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:23.764611 16843 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.765233 16843 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.769824 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:23.785883 16845 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:12:23.787051 16780 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.788676 16780 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.771013 16818 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.792498 16843 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:23.782747 16851 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:23.792560 16818 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.796538 16818 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:23.801578 16469 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.804322 10434 raft_consensus.cc:2368] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:12:23.807163 16852 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.807917 16852 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.810470 16852 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:23.824048 16851 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.814297 16718 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.806779 16469 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:23.858487 16751 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.859275 16751 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.861773 16751 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:23.858497 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.862540 16718 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:23.856606 16843 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.871418 16843 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.870615 16853 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.873782 16828 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.874059 16853 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:23.878052 16853 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:23.882069 16829 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:23.837878 16851 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:23.892863 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 213982us
I20260709 12:12:23.894929 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:12:23.888259 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 118583us
I20260709 12:12:23.899708 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:23.840313 16852 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.888692 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:23.900893 10578 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:23.906256 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:23.950923 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:23.874318 16828 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:23.873818 16843 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:23.953797 16827 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:23.949339 16783 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:23.959854 16829 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.962864 16787 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.973431 16811 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:23.976243 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 228684us
W20260709 12:12:24.001906 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 49726.3us
I20260709 12:12:23.972074 16445 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:23.952937 16852 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.009919 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.012317 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:24.018126 10510 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.008563 16827 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:24.035074 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:24.035861 10580 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.039041 10506 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:24.040212 16859 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:24.052348 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 152762us
I20260709 12:12:24.053076 10578 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:24.009429 16783 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:24.065541 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 69149.1us
I20260709 12:12:24.002667 16843 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:12:24.077786 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:24.078799 10437 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:24.079541 10511 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:24.077849 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.042529 16853 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:24.010149 16811 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:24.006973 16787 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.100124 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.017252 16445 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.060075 16847 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.059755 16859 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.021283 16816 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:12:24.062940 16827 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:24.106070 10438 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:12:24.111351 16783 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.112047 10580 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:24.113812 16783 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.114066 16766 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.114370 16783 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.114670 16766 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:24.115825 16857 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:12:24.117197 16766 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.117386 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.122799 10429 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:12:24.123292 10429 pending_rounds.cc:85] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:12:24.123665 10429 pending_rounds.cc:107] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:12:24.123940 10429 raft_consensus.cc:2889] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:24.124333 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.125725 11178 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:24.123472 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.126799 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.129019 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.133610 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.139956 16873 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.140379 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.140740 16873 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.142706 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.147442 16837 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:24.148268 16837 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.150436 16718 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:24.150935 16718 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.151417 16718 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:24.155160 16838 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.127797 16847 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.155885 16838 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.158265 16838 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.158501 16847 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.155162 16856 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.160331 16856 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.160929 16787 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.162060 16851 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.123327 16854 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:24.172058 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67553.8us
I20260709 12:12:24.107158 16853 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:24.188866 10510 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:24.147496 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.202379 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.202922 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.205044 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:24.195147 10509 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
W20260709 12:12:24.211189 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 85573.1us
W20260709 12:12:24.215526 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117435us
I20260709 12:12:24.114944 16783 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.150876 16837 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.218839 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.226310 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.192050 16856 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:24.291492 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72758.5us
I20260709 12:12:24.292547 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.293666 10580 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
W20260709 12:12:24.298898 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 69288.7us
I20260709 12:12:24.301466 10553 leader_election.cc:400] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:12:24.301895 10553 leader_election.cc:403] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:24.302731 10437 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:12:24.108758 16859 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.305562 11178 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:24.310674 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.291168 16873 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.159509 16827 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.290395 16751 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.300784 16851 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:24.294384 16780 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:24.319682 16783 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:24.320210 16871 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:24.321713 16853 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.323091 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.324923 16859 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.326014 10578 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:24.327325 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:24.331897 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.341380 10585 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
W20260709 12:12:24.332783 16863 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.496s	user 0.007s	sys 0.000s
I20260709 12:12:24.333392 16811 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.337224 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.313719 16353 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:24.369635 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.372339 10437 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:12:24.372823 10437 pending_rounds.cc:85] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 8
I20260709 12:12:24.373081 10437 pending_rounds.cc:107] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:12:24.373323 10437 raft_consensus.cc:2889] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:24.373443 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.374881 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.313081 16787 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:12:24.363994 10587 raft_consensus.cc:2468] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:24.371403 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.325837 16856 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:24.375757 10433 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:24.382709 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.382331 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.377910 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:24.391186 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.390260 16445 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.404023 16878 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.403685 16856 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.404706 16878 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.406476 16856 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.407097 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.407482 10435 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:24.408510 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.394857 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.410051 16877 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:12:24.397734 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.398528 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.382700 16871 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:24.414108 16871 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.362931 16852 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.395282 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.419581 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:24.422833 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 111474us
I20260709 12:12:24.328706 16751 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.340070 16780 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.383148 16353 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:24.443979 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.448246 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.452179 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.425572 16878 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.455577 10507 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:12:24.465209 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.461488 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.467303 10505 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:24.480693 16780 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.490912 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.493535 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:24.499059 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.428666 16851 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:12:24.443305 16828 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.507078 11220 raft_consensus.cc:1275] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:12:24.510076 16867 log.cc:927] Time spent T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260709 12:12:24.501967 16780 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.522622 16870 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:24.535871 16780 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.523191 16353 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.527657 16751 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.541229 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.538266 16780 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.542057 16718 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.513273 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:24.540097 16828 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:24.564883 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.572021 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.574962 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.579993 10513 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:24.580888 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.585292 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:24.586199 10507 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:24.558585 16780 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:24.592010 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:24.592597 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.598431 10404 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.603024 16828 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:24.583154 16870 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:24.593277 16780 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:24.615800 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 365464us
I20260709 12:12:24.617506 16780 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:24.617264 16870 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.644210 11221 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:24.617442 16828 consensus_queue.cc:1048] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:12:24.646173 16870 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.649610 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:24.692102 16718 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.694597 16853 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:24.695277 16853 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.695768 16853 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.712903 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.720320 16811 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:24.732614 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.735332 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.739809 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.748855 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:24.749634 10508 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:24.743248 16843 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:24.738572 16859 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:24.762912 16853 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:24.721081 16811 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.766162 16811 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:24.766955 11220 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:24.790745 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:24.797351 16853 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.797950 10508 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:12:24.836349 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.836952 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.839980 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:24.840651 10433 raft_consensus.cc:3060] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:12:24.841960 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:24.848073 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:24.851480 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.862705 16889 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:24.850906 10433 raft_consensus.cc:2468] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
W20260709 12:12:24.867542 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52916.6us
I20260709 12:12:24.084746  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:24.875465 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.876224 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.876358 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:24.880033 10507 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:24.881996 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.894814 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:24.898259 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.898121 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.902041 11177 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:24.904273 16831 consensus_queue.cc:1059] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:24.910198 16850 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:24.911068 16850 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.913463 16850 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:24.918969 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.882198 16837 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:24.928357 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.931213 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.933832 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.936885 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.938808 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.943135 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:24.918285 16843 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:24.888784 16872 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:12:24.927650 16751 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:24.942291 16856 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:24.944319 16783 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:24.940666 16828 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:24.967212 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.940624 16871 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:24.940786 16846 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:24.963479 16837 raft_consensus.cc:2804] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:12:24.972926 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 84561.8us
I20260709 12:12:24.945300 16851 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:24.920845 16780 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:24.953773 16884 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:24.992105 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:24.940470 16854 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 12:12:24.990361 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 81306us
I20260709 12:12:24.970642 16787 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:25.002976 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 101005us
I20260709 12:12:24.978624 16783 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:25.006085 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 54296.4us
I20260709 12:12:25.019943 16445 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:25.014331 16870 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:12:25.028609 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:25.029472 10435 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:25.025635 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:25.018831 16837 raft_consensus.cc:697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:12:25.055363 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63370.4us
I20260709 12:12:25.025607 16783 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:25.056663 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.056983 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:25.039330 16751 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:25.048672 16856 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:25.074666 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:25.075246 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66014.5us
I20260709 12:12:25.078569 16826 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:25.078773 16780 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:25.079072 16826 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:25.079573 16826 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.084322 16823 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:25.085000 16823 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:25.084443 16902 log.cc:927] Time spent T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:12:25.087179 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:25.085585 16899 log.cc:927] Time spent T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:12:25.091710 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.101485 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.107102 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:25.111436 10504 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:25.072002 16856 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:25.113456 16856 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.086303 16811 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:25.113602 16823 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.117221 16823 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.118810 16851 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:25.071686 16879 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.120462 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:25.122635 16780 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.126197 10429 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:25.126644 16800 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:25.127683 16800 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:25.128144 16800 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:25.128799 16837 consensus_queue.cc:237] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.129186 16800 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.102938 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.127787 10508 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:25.136418 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.139309 16818 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:25.175830 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.190230 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.167604 16853 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:25.173735 16872 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:25.151767 16811 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:12:25.192116 16800 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:25.191627 16828 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:25.174826 16827 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:12:25.271173 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
W20260709 12:12:25.271574 10332 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 233) took 76671 ms (client timeout 15000 ms). Trace:
I20260709 12:12:25.281032 16828 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.283600 16828 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:25.189468 16718 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:12:25.206358 16877 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:25.290268 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:25.272118 10332 rpcz_store.cc:269] 0709 12:11:08.601173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:08.601648 (+   475us) service_pool.cc:224] Handling call
0709 12:11:08.814688 (+213040us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a49090 00007f2c92ae7f3a 00007f2c9889de9c 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc 00007f2c96e5c392 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:25.271448 (+76456760us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:22.877687 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:22.878452 (+   765us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:22.878508 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:12:22.878528 (+    20us) tablet.cc:662] Decoding operations
0709 12:12:22.878955 (+   427us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:22.879333 (+   378us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:22.879369 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:22.879739 (+   370us) tablet.cc:701] Row locks acquired
0709 12:12:22.879794 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:12:22.879978 (+   184us) write_op.cc:270] Start()
0709 12:12:22.880149 (+   171us) write_op.cc:276] Timestamp: P: 1783599142879900 usec, L: 0
0709 12:12:22.880190 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:12:22.880895 (+   705us) log.cc:844] Serialized 535 byte log entry
0709 12:12:23.326751 (+445856us) op_driver.cc:464] REPLICATION: finished
0709 12:12:23.595343 (+268592us) write_op.cc:301] APPLY: starting
0709 12:12:23.601509 (+  6166us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:23.630803 (+ 29294us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:23.630946 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:23.631860 (+   914us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:23.632165 (+   305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:12:23.632207 (+    42us) write_op.cc:312] APPLY: finished
0709 12:12:23.632494 (+   287us) log.cc:844] Serialized 25 byte log entry
0709 12:12:23.677943 (+ 45449us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:23.691154 (+ 13211us) write_op.cc:454] Released schema lock
0709 12:12:23.704324 (+ 13170us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:23.748953 (+ 44629us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:23.750838 (+  1885us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15012,"reported_tablets":52,"spinlock_wait_cycles":279402880,"child_traces":[["op",{"apply.queue_time_us":189617,"apply.run_cpu_time_us":5743,"apply.run_wall_time_us":173603,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":184287,"prepare.run_cpu_time_us":6213,"prepare.run_wall_time_us":10098,"raft.queue_time_us":313328,"raft.run_cpu_time_us":6715,"raft.run_wall_time_us":75414,"replication_time_us":446305,"thread_start_us":547995,"threads_started":4,"wal-append.queue_time_us":80804,"wal-append.run_cpu_time_us":49030,"wal-append.run_wall_time_us":1674426}]]}
W20260709 12:12:25.304337 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 150474us
I20260709 12:12:25.327675 10330 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.329680 16783 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.329813 10330 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 16, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:25.402421 16922 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:12:25.403072 16783 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:25.405318 16783 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.405879 16783 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.403280 10330 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.410012 10330 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.414288 16823 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:25.415280 10330 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 14 to 17, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 17 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.417044 10330 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.418568 10330 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 15, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 15 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.431707 10330 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 17, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 17 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.449951 16445 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:25.449225 10330 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 7 to 10, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.415055 16823 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.492342 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:25.450925 16445 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.504475 10578 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:25.457175 16889 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.531432 16889 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.532217 16445 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:25.466768 16826 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.496122 10429 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:25.533258 10508 raft_consensus.cc:606] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.494170 10580 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:25.533674 10508 raft_consensus.cc:614] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:25.534206 16889 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:25.515357 16783 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:25.536175 10330 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 15 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.537956 10330 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 9 to 14, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 14 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.564311 10330 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 17 to 18, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.534854 16826 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:25.540666 16823 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:25.580493 16804 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:25.581439 16804 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.584342 10508 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:25.583846 10330 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 22, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 22 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.585841 10584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "448b86aa128243e48bed5f471ada874f"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:25.587016 10584 raft_consensus.cc:606] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.608563 10584 raft_consensus.cc:614] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:25.589020 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 83632.5us
I20260709 12:12:25.615124 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:25.463982 10586 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:25.586663 10330 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 11 to 13, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 13 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.621692 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:25.621340 10330 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.646864 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:25.647778 10429 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.648343 10330 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:25.626428 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:25.630452 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:25.645722 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:25.656692 10508 raft_consensus.cc:606] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.661346 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:25.662250 10513 raft_consensus.cc:606] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.662680 10513 raft_consensus.cc:614] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:25.667626 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:25.668609 10513 raft_consensus.cc:606] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.669162 10513 raft_consensus.cc:614] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:25.671289  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:25.674458  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:25.678629  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:25.687882 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:25.696023 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:25.709689 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:25.713011 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:25.713739 10578 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:25.721801 10430 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:12:25.716624 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:25.726040 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:25.741403 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:25.741430 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:25.744189 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:25.748131 10504 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:12:25.742862 10434 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:25.759335 10508 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:25.769250 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:25.770267 10504 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:25.771723 16825 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:25.774083 16850 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:25.774389 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:25.776995 16908 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:25.778043 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:25.774550 16850 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:25.781912 16850 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.783815 16850 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:25.783895 16854 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:25.784452 16854 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.784812 16854 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:25.788204 16914 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:25.794171 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.796077 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.799400 16926 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:25.806865 16856 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:25.807852 16884 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:25.811637 16859 consensus_queue.cc:1059] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:25.771801 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:25.820562 10429 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:25.823851 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:25.825779 10435 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:25.828859 10508 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:25.823370 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:25.819343 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:25.833659 10430 raft_consensus.cc:1275] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:12:25.834293 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:25.841917 10505 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:25.756923 16846 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:25.763681 16818 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:25.740877 16800 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 12:12:25.857522 16804 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:25.841334 16851 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:25.858174 16909 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:25.892910 11214 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:12:25.893520 11214 pending_rounds.cc:85] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 13
I20260709 12:12:25.893765 11214 pending_rounds.cc:107] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:12:25.763012 16886 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 12:12:25.825289 16914 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:12:25.894043 11214 raft_consensus.cc:2889] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:25.879558 16851 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:25.867453 16818 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:25.900533 16882 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:25.915344 16851 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:25.930043 16882 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:25.968925 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:25.975528 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117759us
I20260709 12:12:25.943135 16882 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:25.984735 16882 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:25.994580 10432 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:12:25.976145 16921 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.975708 16849 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:26.071952 10479 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:26.073414 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:26.076397 16823 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:26.076926 16823 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:26.077904 16823 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:26.073529 16826 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:26.111682 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:26.146731 16930 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:12:26.136078 16911 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:12:26.163720 16818 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:26.273468 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:26.306823 10330 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.026s	sys 0.045s
W20260709 12:12:26.307472 10330 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.026s	sys 0.045s
I20260709 12:12:26.164664 16854 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:26.377830 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 73100.2us
W20260709 12:12:26.388134 16931 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.405s	user 0.000s	sys 0.002s
W20260709 12:12:26.406811 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 274267us
W20260709 12:12:26.484470 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78884us
W20260709 12:12:26.403512 16929 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.369s	user 0.001s	sys 0.008s
W20260709 12:12:26.494025 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 465555us
I20260709 12:12:26.483644 16848 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:26.397799 16910 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 12:12:26.513813 16848 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:26.523855 16933 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 12:12:26.524287 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:26.525944 16932 log.cc:927] Time spent T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.316s	user 0.000s	sys 0.000s
W20260709 12:12:26.527590 16935 log.cc:927] Time spent T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 12:12:26.551821 16882 consensus_meta.cc:287] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.000s
I20260709 12:12:26.513949 16908 consensus_queue.cc:1048] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 12:12:26.575690 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 80862.2us
I20260709 12:12:26.583583 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:26.577915 16851 consensus_meta.cc:287] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.003s
W20260709 12:12:26.608698 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 94250.4us
W20260709 12:12:26.602599 16823 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.006s	sys 0.003s
I20260709 12:12:26.601842 16851 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:26.613976 16848 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:26.607718 16854 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:26.612022 16823 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.007s	sys 0.003s
I20260709 12:12:26.622907 16851 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:26.703791 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:26.704985 10584 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:26.715039 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 89550.8us
I20260709 12:12:26.715039 16882 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 12:12:26.719457 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58900.1us
W20260709 12:12:26.740736 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 45669us
I20260709 12:12:26.743700 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:26.721398 16882 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:26.764484 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:26.765239 10509 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:26.779562 10584 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:26.782629 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:26.767669 16957 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:26.789247 10585 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:26.789714 10585 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:26.823863 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:26.824927 11215 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:26.826876 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 2588) took 1084 ms. Trace:
I20260709 12:12:26.827495 10433 rpcz_store.cc:276] 0709 12:12:25.719294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:25.719538 (+   244us) service_pool.cc:224] Handling call
0709 12:12:25.719858 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:25.720234 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:25.720263 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:25.725661 (+  5398us) log.cc:844] Serialized 26 byte log entry
0709 12:12:25.963067 (+237406us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:26.773601 (+810534us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b1800365500. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c926874bc 00007f2c8f9d548b 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9ef455 00007f2c964d73c7 00007f2c92ae99a7 00007f2c92af90bf
0709 12:12:26.773736 (+   135us) log.cc:844] Serialized 14 byte log entry
0709 12:12:26.780666 (+  6930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.780746 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:26.780789 (+    43us) raft_consensus.cc:1685] finished
0709 12:12:26.803356 (+ 22567us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:26.803530 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20767,"prepare.queue_time_us":364510,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":4323,"raft.queue_time_us":108,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":13633,"spinlock_wait_cycles":72154368,"thread_start_us":522028,"threads_started":2,"wal-append.queue_time_us":384192}
I20260709 12:12:26.766014 16882 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:26.872031 16566 raft_consensus.cc:993] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:26.874701 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:26.883760 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 175021us
I20260709 12:12:26.887923 16566 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:26.912745 16566 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.913193 16566 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:26.883494 16927 raft_consensus.cc:1081] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:26.922106 16882 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:26.929862 10509 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:26.930050 10580 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:12:26.976963 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 91131us
W20260709 12:12:26.978600 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 89449.8us
I20260709 12:12:26.977134 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:26.979616 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:26.981040 11212 raft_consensus.cc:2393] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
W20260709 12:12:26.996747 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 220926us
I20260709 12:12:26.992247 16856 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:27.000046 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:27.082336 16566 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:27.088769 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:27.091203 16859 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:27.091919 16859 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:27.092200 10585 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.093575 10585 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:27.094249 16859 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:27.098532 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:27.119032 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41074 (request call id 2908) took 1527 ms. Trace:
I20260709 12:12:27.119760 10507 rpcz_store.cc:276] 0709 12:12:25.576541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:25.577514 (+   973us) service_pool.cc:224] Handling call
0709 12:12:25.578025 (+   511us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:25.578561 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:25.578606 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:25.579649 (+  1043us) log.cc:844] Serialized 52 byte log entry
0709 12:12:25.864689 (+285040us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:25.892499 (+ 27810us) log.cc:844] Serialized 14 byte log entry
0709 12:12:25.892775 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:25.892851 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:26.255544 (+362693us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b180030a300. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:27.045067 (+789523us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b2400167438. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f8789c4 00007f2c8f876acb 00007f2c8fa31dc9 00007f2c8fa31a55 00007f2c8fa30fb0 00007f2c926a7488 00007f2c926a7be8 00007f2c9267f398 00007f2c9267a76a 00007f2c92679bb6
0709 12:12:27.103757 (+ 58690us) raft_consensus.cc:1685] finished
0709 12:12:27.103780 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:27.103948 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27539,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":5000,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":6552,"spinlock_wait_cycles":12169344,"thread_start_us":284431,"threads_started":1,"wal-append.queue_time_us":371597}
I20260709 12:12:27.159018 16566 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:27.162236 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:27.163056 11213 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:27.181017 11213 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:27.097831 16957 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:27.184010 16856 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:27.184012 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:27.192670 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 188797us
I20260709 12:12:27.193825 16957 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.207437 16957 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:27.218509 16482 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:27.229983 16482 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:27.239430 16856 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.240190 16482 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:27.253938 16771 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.258080 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:27.263484 10504 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19.
I20260709 12:12:27.254820 16771 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.291811 16771 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.320304 16771 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:27.320973 16771 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.323395 16771 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:27.331797 16811 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 12:12:27.335769 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 139284us
I20260709 12:12:27.341280 10507 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:27.341734 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:27.349130 16958 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:27.349874 16958 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.350811 16948 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:27.351423 16948 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.353446 16958 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:27.353763 16948 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.390569 16716 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:27.391285 16716 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.393626 16716 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.400527 16850 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:27.401172 16850 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.406350 10431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:27.407052 10431 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:27.407431 10431 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:27.418723 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:27.419569 10504 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:27.446252 16850 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.459553 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.460310 10438 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:27.452948 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.453461 10432 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:27.455682 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.465342 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:27.466096 10585 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:27.466535 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:27.467240 10584 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:27.468302 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:27.470901 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:27.475477 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.476380 10430 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:27.478371 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.456241 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.479854 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.482136 10434 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:27.495913 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:27.503957 10552 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19.
I20260709 12:12:27.504457 10552 leader_election.cc:403] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:12:27.505388 16851 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19.
I20260709 12:12:27.506634 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:27.512840 10552 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:27.512355 10431 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.521802 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:27.513368 16851 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:27.525329 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:27.529657 10431 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.523766 16859 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:27.544716 10479 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:27.533274 16910 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:27.549374 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:27.544298 16859 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:27.551800 16859 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:27.541349 10333 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.566753 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:27.568816 10333 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 10 to 12, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 12 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.569543 10330 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 114) took 4112 ms. Trace:
I20260709 12:12:27.572926 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:27.579875 10509 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:27.580265 10509 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:27.574374 16771 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:27.584296 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:27.585052 10578 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:27.581238 10509 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.592994 10509 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:27.584291 16771 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:27.599378 16771 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:27.590577 16572 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:27.608349 16572 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:27.608914 16572 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:27.592044 10333 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 15 to 16, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.566035 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:27.623150 10509 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
W20260709 12:12:27.623507 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 57570.6us
I20260709 12:12:27.625385 16956 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:27.632373 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:27.633339 16854 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:27.634956 16854 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:27.636062 16854 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:27.648612 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 77544.2us
I20260709 12:12:27.651609 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:27.653394 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:27.582563 10330 rpcz_store.cc:276] 0709 12:12:23.457554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.491119 (+ 33565us) service_pool.cc:224] Handling call
0709 12:12:27.559376 (+4068257us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:26.531311 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:26.531954 (+   643us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:26.531996 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:12:26.532016 (+    20us) tablet.cc:662] Decoding operations
0709 12:12:26.533637 (+  1621us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:26.533708 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:26.533743 (+    35us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:12:26.564668 (+ 30925us) tablet.cc:701] Row locks acquired
0709 12:12:26.564882 (+   214us) write_op.cc:260] PREPARE: finished
0709 12:12:26.565092 (+   210us) write_op.cc:270] Start()
0709 12:12:26.565248 (+   156us) write_op.cc:276] Timestamp: P: 1783599146565030 usec, L: 0
0709 12:12:26.565278 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:12:26.570626 (+  5348us) log.cc:844] Serialized 6308 byte log entry
0709 12:12:27.019981 (+449355us) op_driver.cc:464] REPLICATION: finished
0709 12:12:27.185026 (+165045us) write_op.cc:301] APPLY: starting
0709 12:12:27.185170 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:27.210512 (+ 25342us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:27.210648 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:27.221334 (+ 10686us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:27.221685 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:12:27.221737 (+    52us) write_op.cc:312] APPLY: finished
0709 12:12:27.222157 (+   420us) log.cc:844] Serialized 148 byte log entry
0709 12:12:27.229669 (+  7512us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:27.231673 (+  2004us) write_op.cc:454] Released schema lock
0709 12:12:27.231763 (+    90us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:27.232635 (+   872us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:27.232697 (+    62us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":113685,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":164714,"apply.run_cpu_time_us":22648,"apply.run_wall_time_us":47437,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":829927,"prepare.run_cpu_time_us":70630,"prepare.run_wall_time_us":178483,"raft.queue_time_us":222994,"raft.run_cpu_time_us":12110,"raft.run_wall_time_us":164964,"replication_time_us":454516,"thread_start_us":1058988,"threads_started":4,"wal-append.queue_time_us":181729}]]}
I20260709 12:12:27.660072 10585 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:27.658725 10333 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 9 to 17, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.673727 10478 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:27.670562 16856 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:12:27.694402 10578 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:27.673174 16716 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.703928 16949 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:27.673460 16851 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.687634 16826 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.717466 10586 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:27.733073 10333 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 9 to 10, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 10 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:27.744191 10509 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:27.719892 16859 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.717115 16771 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.749449 16856 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:12:27.762584 16859 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:27.765044 16771 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.767063 10333 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 9 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.768743 16572 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:27.769191 10333 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 12 to 16, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:27.776156 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:27.798266 10429 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:27.770947 16572 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:27.808092  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:27.849604 10333 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 11 to 18, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 18 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.861310 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:27.862385 10433 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:27.867094 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:27.868039 10508 raft_consensus.cc:2368] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:12:27.871675 10333 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 9 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:27.880237 16927 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:27.881080 16927 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:27.874164 10333 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 14 to 15, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:27.905041 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:27.905566 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:27.906067 10584 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:27.892501 10586 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:12:27.965812 10433 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:27.948271 16927 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:27.979349 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:27.991251 10430 raft_consensus.cc:2393] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:27.985534 10584 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:27.993080 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:27.994603 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 148202us
I20260709 12:12:28.003177 10552 leader_election.cc:400] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:12:28.003536 10552 leader_election.cc:403] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:12:28.007731 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 120319us
I20260709 12:12:28.010326 16859 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
W20260709 12:12:28.081703 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92325.5us
W20260709 12:12:28.154321 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 117822us
W20260709 12:12:28.099303 16991 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:12:28.163476 16989 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:12:28.207265 10580 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:28.100209 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:28.231174 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 131079us
W20260709 12:12:28.248072 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58429.4us
W20260709 12:12:28.265878 16990 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 12:12:28.273134 16995 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:12:28.288264 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:28.291131 16854 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:28.248306 16850 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:28.310964 10579 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
W20260709 12:12:28.324462 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 231781us
I20260709 12:12:28.335623 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:28.341310 10511 raft_consensus.cc:606] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:28.341867 10511 raft_consensus.cc:614] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:28.324105 16850 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:28.348675 16850 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.308367 17006 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.121s
W20260709 12:12:28.392022 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 197050us
W20260709 12:12:28.389016 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 231069us
I20260709 12:12:28.401948 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:28.402755 10511 raft_consensus.cc:606] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:28.408749 16948 raft_consensus.cc:993] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:28.411085 11213 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:12:28.416867 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:28.437259 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 195331us
I20260709 12:12:28.411831 16948 raft_consensus.cc:1081] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:28.456144 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:28.460738 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:28.461457 10511 raft_consensus.cc:606] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:28.463790 16858 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:28.464480 16858 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:28.465528 16858 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:28.489130 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96261.1us
I20260709 12:12:28.496798 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:28.502847 16917 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:28.503628 16826 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:28.503651 16917 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.507468 16917 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:28.507761 17017 tablet_replica.cc:333] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:12:28.509074 17017 raft_consensus.cc:2243] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:28.509847 17017 raft_consensus.cc:2272] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:28.514889 10509 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:28.497303 16832 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:28.523650 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:28.524355 10509 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:28.524812 10509 raft_consensus.cc:614] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:28.527804 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:28.536119 16927 raft_consensus.cc:993] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:28.536752 16927 raft_consensus.cc:1081] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:28.556787 10433 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:28.563234 10433 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.563941 10433 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:28.583223 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 126561us
I20260709 12:12:28.598620 10430 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:28.599393 10430 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.599905 10430 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:28.611196 16998 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:28.613854 16998 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.614378 16998 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.626027 16998 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:28.627305 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:28.633802 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:28.574398 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:28.634709 10578 raft_consensus.cc:2410] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:12:28.634845 10509 raft_consensus.cc:606] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:28.635290 10509 raft_consensus.cc:614] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:28.636807 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47830.5us
I20260709 12:12:28.643411 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:28.647424 10430 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.650787  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:28.652328 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:28.655922  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:28.668056 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:28.668608 10509 raft_consensus.cc:3055] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:28.668872 10509 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:28.669126 10430 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:28.670271 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:28.672905 10585 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:28.676563 10509 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.677589 10509 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:28.671671 10433 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.680233 10434 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:28.686312 10585 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:12:28.686625 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:28.658499 16921 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:28.690565 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:28.691248 10506 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:28.691260 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:28.691717 10506 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:28.692140 10585 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:28.689909 16956 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:28.692595 10506 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.699862 10506 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:28.702531 10509 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:12:28.688360 10433 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:28.689599 10403 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:28.711930 10585 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:28.716651 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:28.709084 17017 ts_tablet_manager.cc:1916] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:28.721383 16998 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:28.731534 16998 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:28.732689 16998 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.720072 16921 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:28.759727 16921 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:28.779491 16921 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:28.796130 10429 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:28.813501 16949 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:28.822752  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:28.848940 10506 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:28.895961 10581 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:28.911123 16949 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:28.982800 17017 ts_tablet_manager.cc:1929] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.9
I20260709 12:12:28.988207 17017 log.cc:1199] T deab11eaf55b4f3ebe48926a1e2afa38 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/deab11eaf55b4f3ebe48926a1e2afa38
I20260709 12:12:29.018766 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet deab11eaf55b4f3ebe48926a1e2afa38 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:12:29.032343 10333 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 238) took 25903 ms (client timeout 15000 ms). Trace:
I20260709 12:12:29.038019 10431 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:29.041397 16858 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:12:29.033077 10333 rpcz_store.cc:269] 0709 12:12:03.121592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.123627 (+  2035us) service_pool.cc:224] Handling call
0709 12:12:29.024638 (+25901011us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:28.166461 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:28.167185 (+   724us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:28.167229 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:12:28.167243 (+    14us) tablet.cc:662] Decoding operations
0709 12:12:28.168378 (+  1135us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:28.168444 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:28.168484 (+    40us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:12:28.169449 (+   965us) tablet.cc:701] Row locks acquired
0709 12:12:28.169466 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:12:28.169664 (+   198us) write_op.cc:270] Start()
0709 12:12:28.169820 (+   156us) write_op.cc:276] Timestamp: P: 1783599148169603 usec, L: 0
0709 12:12:28.169839 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:12:28.170632 (+   793us) log.cc:844] Serialized 3974 byte log entry
0709 12:12:28.671158 (+500526us) op_driver.cc:464] REPLICATION: finished
0709 12:12:28.748159 (+ 77001us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b4400022b4c. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f918c85 00007f2c8f9196a2 00007f2c9268b1db 00007f2c92686aaf 00007f2c8f9d50ac 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c96515ef0
0709 12:12:28.812667 (+ 64508us) write_op.cc:301] APPLY: starting
0709 12:12:28.840011 (+ 27344us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:28.877982 (+ 37971us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:28.878008 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:28.892969 (+ 14961us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:28.893359 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:12:28.893408 (+    49us) write_op.cc:312] APPLY: finished
0709 12:12:28.893749 (+   341us) log.cc:844] Serialized 97 byte log entry
0709 12:12:28.893977 (+   228us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:28.898956 (+  4979us) write_op.cc:454] Released schema lock
0709 12:12:28.899926 (+   970us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:28.917982 (+ 18056us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:28.918020 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":132961,"apply.run_cpu_time_us":16578,"apply.run_wall_time_us":110693,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":166492,"prepare.run_cpu_time_us":4718,"prepare.run_wall_time_us":76588,"replication_time_us":479105,"spinlock_wait_cycles":11518336,"thread_start_us":187366,"threads_started":2}]]}
I20260709 12:12:29.072283 10581 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:29.077077 16927 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:12:29.109261 10463 heartbeater.cc:646] Failed to heartbeat to 127.9.11.126:42677 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.11.126:42677 timed out after 15.000s (SENT)
I20260709 12:12:29.134836 10334 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 11 to 12, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 12 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:29.235368 10581 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:12:29.240418 16856 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:29.235543 10334 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 8 to 11, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:29.264889 10506 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:12:29.278558 16911 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:12:29.285986 10334 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:29.292296 10507 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:29.300549 16856 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:29.388013 10581 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:29.412714 16854 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:12:29.610633 10330 catalog_manager.cc:5697] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 19 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:29.613606 10330 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 19 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:29.653066 10330 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 19 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:29.655130 10330 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:29.670812 10330 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:29.713171 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:33776
W20260709 12:12:29.712431 10334 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 235) took 41221 ms (client timeout 15000 ms). Trace:
I20260709 12:12:29.729326 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet deab11eaf55b4f3ebe48926a1e2afa38 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:29.747635 10330 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 115) took 1972 ms. Trace:
W20260709 12:12:29.731640 10334 rpcz_store.cc:269] 0709 12:11:48.491055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.495934 (+  4879us) service_pool.cc:224] Handling call
0709 12:11:49.195598 (+699664us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b5400270010. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c8f87adb1 00007f2c8f9bab71 00007f2c8f9bb0b2 00007f2c98934759 00007f2c988cca76 00007f2c9889dd58 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc
0709 12:12:29.712335 (+40516737us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:29.413932 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:29.414614 (+   682us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:29.414650 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:12:29.414668 (+    18us) tablet.cc:662] Decoding operations
0709 12:12:29.415307 (+   639us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:29.415389 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:29.415416 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:12:29.415781 (+   365us) tablet.cc:701] Row locks acquired
0709 12:12:29.415799 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:12:29.415950 (+   151us) write_op.cc:270] Start()
0709 12:12:29.416071 (+   121us) write_op.cc:276] Timestamp: P: 1783599149415895 usec, L: 0
0709 12:12:29.416096 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:12:29.416892 (+   796us) log.cc:844] Serialized 1295 byte log entry
0709 12:12:29.524238 (+107346us) op_driver.cc:464] REPLICATION: finished
0709 12:12:29.563572 (+ 39334us) write_op.cc:301] APPLY: starting
0709 12:12:29.563686 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:29.569609 (+  5923us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:29.569702 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:29.581900 (+ 12198us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:29.582133 (+   233us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:12:29.582183 (+    50us) write_op.cc:312] APPLY: finished
0709 12:12:29.582424 (+   241us) log.cc:844] Serialized 41 byte log entry
0709 12:12:29.582610 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:29.582720 (+   110us) write_op.cc:454] Released schema lock
0709 12:12:29.582766 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:29.583308 (+   542us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:29.583341 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1129164,"reported_tablets":52,"spinlock_wait_cycles":79129984,"child_traces":[["op",{"apply.queue_time_us":36351,"apply.run_cpu_time_us":6291,"apply.run_wall_time_us":19692,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":75873,"prepare.run_cpu_time_us":3383,"prepare.run_wall_time_us":10678,"replication_time_us":107934,"thread_start_us":90021,"threads_started":2}]]}
I20260709 12:12:29.769618 10330 rpcz_store.cc:276] 0709 12:12:27.775349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:27.797281 (+ 21932us) service_pool.cc:224] Handling call
0709 12:12:29.747525 (+1950244us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:29.675938 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:29.676582 (+   644us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:29.676625 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:12:29.676646 (+    21us) tablet.cc:662] Decoding operations
0709 12:12:29.677408 (+   762us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:29.677480 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:29.677519 (+    39us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:12:29.678377 (+   858us) tablet.cc:701] Row locks acquired
0709 12:12:29.678395 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:12:29.678593 (+   198us) write_op.cc:270] Start()
0709 12:12:29.678727 (+   134us) write_op.cc:276] Timestamp: P: 1783599149678535 usec, L: 0
0709 12:12:29.678751 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:12:29.679521 (+   770us) log.cc:844] Serialized 2039 byte log entry
0709 12:12:29.690370 (+ 10849us) op_driver.cc:464] REPLICATION: finished
0709 12:12:29.690936 (+   566us) write_op.cc:301] APPLY: starting
0709 12:12:29.691053 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:29.694320 (+  3267us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:29.694423 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:29.699368 (+  4945us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:29.699632 (+   264us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:12:29.699690 (+    58us) write_op.cc:312] APPLY: finished
0709 12:12:29.699965 (+   275us) log.cc:844] Serialized 57 byte log entry
0709 12:12:29.700922 (+   957us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:29.701067 (+   145us) write_op.cc:454] Released schema lock
0709 12:12:29.701119 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:29.701620 (+   501us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:29.701659 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30229,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":298,"apply.run_cpu_time_us":8656,"apply.run_wall_time_us":10628,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20910,"num_ops":5,"prepare.queue_time_us":144,"prepare.run_cpu_time_us":3796,"prepare.run_wall_time_us":3932,"replication_time_us":11448}]]}
I20260709 12:12:29.805686 10330 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:29.992117 10328 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:29.994382 10328 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.007226 10328 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.012245 10328 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 14, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } }
I20260709 12:12:30.015203 10328 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.023066 10328 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 14, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.025314 10328 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 16, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.030717 10328 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.065258 10328 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.067747 10328 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 10 to 13, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.071143 10328 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 17 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.092258 10328 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.203262  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:30.294220 10333 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 17 to 18, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:30.297204 10333 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 17, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 17 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:30.299351 10333 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 19, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.300832 10333 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.302379 10333 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.311918 10333 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:30.313973 10328 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 226) took 16179 ms (client timeout 15000 ms). Trace:
I20260709 12:12:30.321045 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet e4cd5d330f694dacb199bd61266c8f5a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:51568
I20260709 12:12:30.314494 10333 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 19 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:30.333375 10333 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.337361 10333 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 14, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.344934 10333 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 17 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.347713 10333 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.365723 10333 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 17, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 17 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:30.371857 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet e4cd5d330f694dacb199bd61266c8f5a (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:30.379660 10333 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.381544 10333 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.383230 10333 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.384527 10333 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.385974 10333 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
W20260709 12:12:30.314551 10328 rpcz_store.cc:269] 0709 12:12:14.135053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.212452 (+ 77399us) service_pool.cc:224] Handling call
0709 12:12:30.313870 (+16101418us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:30.103009 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:30.103713 (+   704us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:30.103760 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:12:30.103780 (+    20us) tablet.cc:662] Decoding operations
0709 12:12:30.105197 (+  1417us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:30.105278 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:30.105315 (+    37us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:12:30.115060 (+  9745us) tablet.cc:701] Row locks acquired
0709 12:12:30.115089 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:12:30.115303 (+   214us) write_op.cc:270] Start()
0709 12:12:30.115461 (+   158us) write_op.cc:276] Timestamp: P: 1783599150115252 usec, L: 0
0709 12:12:30.115498 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:12:30.116353 (+   855us) log.cc:844] Serialized 4734 byte log entry
0709 12:12:30.118661 (+  2308us) op_driver.cc:464] REPLICATION: finished
0709 12:12:30.119200 (+   539us) write_op.cc:301] APPLY: starting
0709 12:12:30.119308 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:30.146247 (+ 26939us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:30.146375 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:30.171725 (+ 25350us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:30.172037 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:12:30.172091 (+    54us) write_op.cc:312] APPLY: finished
0709 12:12:30.172467 (+   376us) log.cc:844] Serialized 113 byte log entry
0709 12:12:30.172683 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:30.172867 (+   184us) write_op.cc:454] Released schema lock
0709 12:12:30.172930 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:30.187697 (+ 14767us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:30.187730 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14102,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":282,"apply.run_cpu_time_us":18473,"apply.run_wall_time_us":68444,"cfile_cache_hit":48,"cfile_cache_hit_bytes":50184,"num_ops":12,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":5452,"prepare.run_wall_time_us":13956,"replication_time_us":2951}]]}
I20260709 12:12:30.394428 10333 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 13 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:30.396133 10333 catalog_manager.cc:5697] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.399415 10333 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 8 to 17, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.404592 10333 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.416026 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:30.416854 10506 raft_consensus.cc:606] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:30.433837 10333 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 17 to 20, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:30.435626 10333 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 9 to 12, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:30.437058 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:30.437795 10506 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:30.444479 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:30.445212 10506 raft_consensus.cc:606] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:30.489127 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:30.489856 10506 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:30.492887 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:30.493515 10506 raft_consensus.cc:606] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:30.496869 10506 raft_consensus.cc:614] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:30.498088  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:30.500667  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:30.501654  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:30.582135 17040 raft_consensus.cc:993] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.582737 17040 raft_consensus.cc:1081] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.584811 10431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:30.585460 10431 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:30.585852 10431 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:30.609992 10431 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:30.612332 10431 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:30.613942 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:30.614738 10587 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:30.619459 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:30.620223 10506 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:30.620538 10506 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:30.641222 10506 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:30.642896 10506 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:30.644600 17040 raft_consensus.cc:993] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.645193 16927 raft_consensus.cc:1081] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.647197 10431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:30.647864 10431 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:30.648264 10431 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:30.657861 10431 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:30.676781 10431 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:30.692101 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:30.692830 10509 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:30.693178 10509 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:30.694051 10509 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:30.699713 10509 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:30.714775 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:30.735646 10578 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:30.835536 17040 raft_consensus.cc:993] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.836192 17040 raft_consensus.cc:1081] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.838145 10436 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:30.838784 10436 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:30.839367 10436 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:30.849895 10587 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:30.854329 10436 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:30.872061 10509 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:30.874372 10506 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:30.876972 10436 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:30.873904 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:30.895833 17040 raft_consensus.cc:993] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.909088 17040 raft_consensus.cc:1081] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:30.909369 17043 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:30.910027 17043 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:30.911079 17043 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:30.915545 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:30.917629 17042 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:30.922761 17042 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:30.923786 17042 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:30.927201 10431 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:30.927889 10431 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:30.928282 10431 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:30.929925 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:30.930578 10506 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:30.926363 10578 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:30.956918 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:30.957656 10585 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:30.930881 10506 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:30.974041 10506 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:30.975700 10506 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:31.001353 10431 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:31.017448 10585 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:31.026310 10431 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:31.060978 10506 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:31.110654 10333 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 227) took 1796 ms. Trace:
I20260709 12:12:31.114710 10333 rpcz_store.cc:276] 0709 12:12:29.314048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.314344 (+   296us) service_pool.cc:224] Handling call
0709 12:12:31.110565 (+1796221us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:30.497463 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:30.498124 (+   661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:30.498167 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:12:30.498190 (+    23us) tablet.cc:662] Decoding operations
0709 12:12:30.501761 (+  3571us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:30.501837 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:30.501872 (+    35us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:12:30.503927 (+  2055us) tablet.cc:701] Row locks acquired
0709 12:12:30.503950 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:12:30.504128 (+   178us) write_op.cc:270] Start()
0709 12:12:30.504270 (+   142us) write_op.cc:276] Timestamp: P: 1783599150504072 usec, L: 0
0709 12:12:30.504298 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:12:30.505314 (+  1016us) log.cc:844] Serialized 8936 byte log entry
0709 12:12:30.509011 (+  3697us) op_driver.cc:464] REPLICATION: finished
0709 12:12:30.509540 (+   529us) write_op.cc:301] APPLY: starting
0709 12:12:30.509633 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:30.531552 (+ 21919us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:30.531663 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:30.644356 (+112693us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:30.644792 (+   436us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 12:12:30.644841 (+    49us) write_op.cc:312] APPLY: finished
0709 12:12:30.655526 (+ 10685us) log.cc:844] Serialized 204 byte log entry
0709 12:12:30.659412 (+  3886us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:30.659676 (+   264us) write_op.cc:454] Released schema lock
0709 12:12:30.659759 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:30.673496 (+ 13737us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:30.673536 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":89580,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":293,"apply.run_cpu_time_us":33052,"apply.run_wall_time_us":163898,"cfile_cache_hit":92,"cfile_cache_hit_bytes":96186,"num_ops":23,"prepare.queue_time_us":14550,"prepare.run_cpu_time_us":6650,"prepare.run_wall_time_us":11188,"replication_time_us":4545}]]}
W20260709 12:12:31.127130 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 102287us
I20260709 12:12:31.136032 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 239) took 8807 ms. Trace:
I20260709 12:12:31.129071 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:31.136554 10336 rpcz_store.cc:276] 0709 12:12:22.328450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.328746 (+   296us) service_pool.cc:224] Handling call
0709 12:12:31.135945 (+8807199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175130,"reported_tablets":53}
W20260709 12:12:31.231415 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49809.9us
I20260709 12:12:31.187924 17042 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:31.234432 17042 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:31.242822 17042 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:12:31.285032 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 156049us
I20260709 12:12:31.296988 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:31.297784 10507 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:31.298130 10507 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:31.298540 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:31.299515 10585 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:31.299327 10507 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:31.301424 10507 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:31.350507 10507 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:12:31.398072 10585 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
W20260709 12:12:31.401608 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 72006.9us
W20260709 12:12:31.403918 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 85633.5us
I20260709 12:12:31.484089 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:31.530872 17043 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:31.531884 17043 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:31.532954 17043 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:31.536653 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 52674.9us
I20260709 12:12:31.565289 10578 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:31.581204 10505 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:31.638270 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:31.642102 10507 consensus_queue.cc:237] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:31.657599 10505 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:31.667361 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 237) took 4207 ms. Trace:
I20260709 12:12:31.668028 10332 rpcz_store.cc:276] 0709 12:12:27.459953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:27.460198 (+   245us) service_pool.cc:224] Handling call
0709 12:12:31.667241 (+4207043us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54967,"reported_tablets":52}
I20260709 12:12:31.670157 11212 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:31.672220 17051 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:12:31.676863 17059 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:12:31.682925 10585 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:31.705502 10431 raft_consensus.cc:1275] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:31.707360 17040 consensus_queue.cc:1048] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:31.798833 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet deab11eaf55b4f3ebe48926a1e2afa38 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:12:31.803741 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 65653.5us
I20260709 12:12:31.848233 17069 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 7 to 14, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:31.890897 16921 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:12:31.889503 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 80578.1us
I20260709 12:12:31.904171 10336 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 20 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:31.944429 10585 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:31.936726  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:31.963181 17043 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:12:31.966845 10336 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:31.985859 17071 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:31.989398 11215 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:32.019085 11212 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 7 to 14, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:32.052965 17040 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:32.059279 10431 raft_consensus.cc:2955] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 7 to 14, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:32.150962 10587 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:32.260742 10507 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:32.284447 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81b70ca03fbe4622a03717efd079d598 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:32.311594 10513 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:32.318238 10328 catalog_manager.cc:5697] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 7 to 14, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:32.372471 10329 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 236) took 21475 ms (client timeout 15000 ms). Trace:
I20260709 12:12:32.371886 17071 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 11 to 13, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:32.381999 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet deab11eaf55b4f3ebe48926a1e2afa38 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:33776
W20260709 12:12:32.373080 10329 rpcz_store.cc:269] 0709 12:12:10.897300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.904459 (+  7159us) service_pool.cc:224] Handling call
0709 12:12:32.372171 (+21467712us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36680,"reported_tablets":52}
I20260709 12:12:32.412643 11213 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 11 to 13, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:32.440021 10579 raft_consensus.cc:2955] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 11 to 13, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:32.457154 10508 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:32.464764 10585 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:32.484678 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cfb1df1b6ba3414899a137b410511b26 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:32.504040 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet deab11eaf55b4f3ebe48926a1e2afa38 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:32.516049 17081 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:32.520184 17043 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:32.539508 17080 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:12:32.580098 17081 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:32.605963 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65184.3us
I20260709 12:12:32.606062 17043 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
W20260709 12:12:32.788980 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59870.2us
W20260709 12:12:32.923867 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 267676us
W20260709 12:12:32.956125 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44996.9us
W20260709 12:12:32.971688 17100 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.116s	user 0.000s	sys 0.005s
I20260709 12:12:32.978753 17118 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:32.979894 17118 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:32.987967 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 228) took 1725 ms. Trace:
I20260709 12:12:32.988610 10336 rpcz_store.cc:276] 0709 12:12:31.275132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.275339 (+   207us) service_pool.cc:224] Handling call
0709 12:12:32.987845 (+1712506us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:32.154457 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:32.190118 (+ 35661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:32.190175 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:12:32.190210 (+    35us) tablet.cc:662] Decoding operations
0709 12:12:32.190755 (+   545us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:32.190850 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:32.190877 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:12:32.215868 (+ 24991us) tablet.cc:701] Row locks acquired
0709 12:12:32.215928 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:12:32.216112 (+   184us) write_op.cc:270] Start()
0709 12:12:32.216288 (+   176us) write_op.cc:276] Timestamp: P: 1783599152216041 usec, L: 0
0709 12:12:32.216332 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:12:32.217836 (+  1504us) log.cc:844] Serialized 917 byte log entry
0709 12:12:32.535350 (+317514us) op_driver.cc:464] REPLICATION: finished
0709 12:12:32.610282 (+ 74932us) write_op.cc:301] APPLY: starting
0709 12:12:32.610411 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:32.613540 (+  3129us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:32.613680 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:32.641261 (+ 27581us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:32.670403 (+ 29142us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:12:32.670521 (+   118us) write_op.cc:312] APPLY: finished
0709 12:12:32.670886 (+   365us) log.cc:844] Serialized 33 byte log entry
0709 12:12:32.671167 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:32.671296 (+   129us) write_op.cc:454] Released schema lock
0709 12:12:32.671376 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:32.671953 (+   577us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:32.671997 (+    44us) write_op.cc:454] Released schema lock
0709 12:12:32.944960 (+272963us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:32.949566 (+  4606us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":476538,"reported_tablets":4,"spinlock_wait_cycles":9984,"child_traces":[["op",{"apply.queue_time_us":74531,"apply.run_cpu_time_us":6715,"apply.run_wall_time_us":61642,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":122505,"prepare.run_cpu_time_us":21924,"prepare.run_wall_time_us":129944,"raft.queue_time_us":164928,"raft.run_cpu_time_us":45199,"raft.run_wall_time_us":404684,"replication_time_us":318812,"thread_start_us":222343,"threads_started":4,"wal-append.queue_time_us":161211}]]}
I20260709 12:12:33.026463 17118 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:33.032714 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 162845us
W20260709 12:12:33.091347 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 148104us
I20260709 12:12:33.108108 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:33.117689 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 81b70ca03fbe4622a03717efd079d598 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:55454
W20260709 12:12:33.238682 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 112068us
I20260709 12:12:33.261139 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:33.307597 17131 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:33.308578 17131 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:33.309376 17101 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.519s	user 0.003s	sys 0.010s
I20260709 12:12:33.316212 17131 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:33.341267 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:33.336792 10552 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:33.344233 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 229321us
I20260709 12:12:33.356161 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:33.358949 17128 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:33.382723 17136 tablet_replica.cc:333] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:12:33.411120 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:33.415889 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:33.420290 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 65273.3us
W20260709 12:12:33.445462 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 82819.2us
I20260709 12:12:33.477648 10336 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 11 to 13, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:33.482177 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:33.483093 10508 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:33.487641 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "448b86aa128243e48bed5f471ada874f"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:33.488277 10508 raft_consensus.cc:606] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:33.491590 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:33.492151 10508 raft_consensus.cc:606] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:33.495038 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:33.495651 10508 raft_consensus.cc:606] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:33.498389 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:33.498971 10508 raft_consensus.cc:606] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:33.500478  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:12:33.500602 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 145168us
I20260709 12:12:33.503019  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:12:33.520524 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 72188.7us
I20260709 12:12:33.533298 17136 raft_consensus.cc:2243] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:33.535147 17136 raft_consensus.cc:2272] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:33.544549 17136 ts_tablet_manager.cc:1916] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:33.550743 17130 raft_consensus.cc:993] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.551564 17130 raft_consensus.cc:1081] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.555512  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:33.579749 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:33.580552 10581 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:33.580940 10581 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.582762 10403 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:33.588827 17115 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:33.615542 10581 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:33.642524 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:33.643290 10513 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:33.643605 10513 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:33.644533 10513 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:33.645841 10513 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.651244 10581 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:33.655265 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet cfb1df1b6ba3414899a137b410511b26 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:51568
I20260709 12:12:33.662874 10513 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:33.672189 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:33.673311 10430 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.680490 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:33.707491 17139 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:33.708174 17139 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:33.709195 17139 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:33.765807 10430 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:33.774561 10328 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 15 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:33.778056 17088 raft_consensus.cc:993] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.777415 10328 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 13, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 13 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:33.778676 17088 raft_consensus.cc:1081] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.790426 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:33.791169 10581 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:33.791714 10581 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.804733 17151 tablet_replica.cc:333] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:12:33.806380 17151 raft_consensus.cc:2243] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:33.812970 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 238) took 1646 ms. Trace:
I20260709 12:12:33.813632 10332 rpcz_store.cc:276] 0709 12:12:32.166670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.166866 (+   196us) service_pool.cc:224] Handling call
0709 12:12:33.812761 (+1645895us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93305,"reported_tablets":10,"spinlock_wait_cycles":2432}
I20260709 12:12:33.847739 17088 raft_consensus.cc:993] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.848524 17082 raft_consensus.cc:1081] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.854828 10585 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:33.855499 10585 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:33.855913 10585 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:33.860503 17151 raft_consensus.cc:2272] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:33.853739 17136 ts_tablet_manager.cc:1929] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.13
I20260709 12:12:33.863265 17136 log.cc:1199] T 81b70ca03fbe4622a03717efd079d598 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/81b70ca03fbe4622a03717efd079d598
I20260709 12:12:33.884016 10581 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:33.877463 10585 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:33.867151 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 81b70ca03fbe4622a03717efd079d598 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:33.895859 10585 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:33.896850 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:33.897562 10513 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:33.897895 10513 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:33.899353 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:33.899890 10430 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:33.901384 10513 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:33.906018 17151 ts_tablet_manager.cc:1916] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:33.933355 10581 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:33.935176 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:33.936182 10508 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:33.936399 10508 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:33.937655 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:33.939208 11212 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.937804 10508 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:33.946408 10508 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.962451 10430 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:33.986756 11212 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:33.988981 10513 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:33.988168 10553 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:33.992609 17069 raft_consensus.cc:993] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.993199 17069 raft_consensus.cc:1081] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.995169 17140 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:33.995749 17140 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:33.996814 17140 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:33.995803 17088 raft_consensus.cc:993] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:33.999574 17088 raft_consensus.cc:1081] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:34.005163 10583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "448b86aa128243e48bed5f471ada874f"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:34.005813 10583 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.006170 10583 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:34.006032 10587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:34.011626 10587 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.012059 10587 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.050585 10508 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:34.073376 10333 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 240) took 2482 ms. Trace:
I20260709 12:12:34.073957 10333 rpcz_store.cc:276] 0709 12:12:31.591133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.591328 (+   195us) service_pool.cc:224] Handling call
0709 12:12:34.073099 (+2481771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55633,"reported_tablets":23}
I20260709 12:12:34.078963 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 81b70ca03fbe4622a03717efd079d598 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 14)) from {username='slave'} at 127.0.0.1:55454
W20260709 12:12:34.080758 10335 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 236) took 81050 ms (client timeout 15000 ms). Trace:
W20260709 12:12:34.081421 10335 rpcz_store.cc:269] 0709 12:11:13.030641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.030934 (+   293us) service_pool.cc:224] Handling call
0709 12:11:13.660634 (+629700us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a49090 00007f2c92ae7f3a 00007f2c9889de9c 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc 00007f2c96e5c392 00007f2c90e1a48c 00007f2c90e198c6
0709 12:12:34.080504 (+80419870us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170513,"reported_tablets":52,"spinlock_wait_cycles":6374656}
I20260709 12:12:34.084393 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 81b70ca03fbe4622a03717efd079d598 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:34.085219 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 0b474d9b241d4a698702287ee1bbe3b2 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:55454
I20260709 12:12:34.087352 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 0b474d9b241d4a698702287ee1bbe3b2 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:34.129261 17014 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:34.130055 17014 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.132678 17014 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:34.134312 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:34.146916 10583 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.157042 10583 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:34.159462 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:34.160257 10508 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:34.160597 10508 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:34.161468 10508 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.165552 10513 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:34.166958 10552 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:34.142589 10587 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.162968 10508 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:34.172415 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 166511us
I20260709 12:12:34.178849 10587 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:34.179989 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:34.180644 10513 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:34.180956 10513 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:34.181789 10513 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.186091 10513 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.191026 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:34.179852 17139 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:34.196707 10511 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:34.196831 10436 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:34.196713 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:34.199783 17139 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:34.220041 17014 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:34.223803 17139 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.257781 10508 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:12:34.257674 10513 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:34.272282 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:34.273854 10552 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:34.274806 17014 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:34.275350 16956 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:34.272289 10333 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:34.277519 10333 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:34.287631 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:34.261584 10436 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:12:34.292891 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:34.293515 10430 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.297823 17014 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:34.302721 17155 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:34.305326 17014 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.325603 10430 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:34.328305 10433 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:34.345245 16956 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:34.348722 17140 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:34.346819 16956 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:34.375344 17151 ts_tablet_manager.cc:1929] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.12
I20260709 12:12:34.401773 17151 log.cc:1199] T cfb1df1b6ba3414899a137b410511b26 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/cfb1df1b6ba3414899a137b410511b26
I20260709 12:12:34.415959 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet cfb1df1b6ba3414899a137b410511b26 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
W20260709 12:12:34.421629 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56888.1us
I20260709 12:12:34.462944 10511 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:12:34.465726 17153 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:12:34.598436 11213 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:12:34.601298 17163 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:12:34.605039 10333 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 14 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:34.607386 10333 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 20 to 21, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 21 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:34.609058 10333 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 16 to 17, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:34.674093 10511 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:34.688272 17163 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:34.728876 10436 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:12:34.746485 10436 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:34.751984 17161 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:34.759653 17160 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:34.789568  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:34.830302 10508 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:34.843649 17160 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:34.901371 10429 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:34.905925 10508 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:12:34.906864 17155 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:34.925395 16956 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:12:35.173506 10430 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:35.185624 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:35.186241 10429 raft_consensus.cc:606] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:35.189385 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:35.190155 10429 raft_consensus.cc:606] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:35.208994 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:35.209769 10581 raft_consensus.cc:606] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:35.217029 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:35.219079 10429 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:35.220705 17185 raft_consensus.cc:993] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:35.222301 17110 raft_consensus.cc:1081] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:35.225248 10513 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:12:35.226246 10404 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fa6e3eaefd14aff89099fd64eeb1a1d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:35.227475 10587 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:12:35.236886 10513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:35.237574  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:35.237689 10513 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:35.238291 10513 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:35.240338  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:35.242712  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:35.291745 17115 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:35.297116 17185 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:35.331667 10513 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:35.334017 10513 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:35.335601 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:35.336298 10431 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:35.336405 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:35.336663 10431 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:35.337157 11213 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:35.337724 10431 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:35.338958 10431 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:35.369362 10431 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:35.376063 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:35.377770 17137 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:35.378371 17137 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:35.379478 17137 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:35.387125 17159 raft_consensus.cc:993] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:35.387924 16956 raft_consensus.cc:1081] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
W20260709 12:12:35.395876 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50211.3us
I20260709 12:12:35.402679 10511 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:35.403294 10511 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:35.403687 10511 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:35.414119 10511 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.434015 10511 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:35.433194 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:35.443269 10433 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:35.443246 11213 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:35.444408 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:35.445200 10587 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:35.445516 10587 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:35.452280 10587 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.467471 17131 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:35.469419 10587 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:35.481628 10433 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:35.494747 10431 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:35.489670 10328 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 15 to 16, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:35.495642 10511 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:35.486203 10587 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
W20260709 12:12:35.507962 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 61500.3us
I20260709 12:12:35.509004 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:35.516024 17185 raft_consensus.cc:993] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:35.533188 17130 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:35.516781 17185 raft_consensus.cc:1081] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:35.533947 17130 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:35.537158 17130 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.564792 10335 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:35.578964 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 70007.5us
I20260709 12:12:35.583016 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:12:35.582681 10587 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:35.583611 10581 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:35.584882 10581 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:35.598838 17185 raft_consensus.cc:993] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:35.599507 17185 raft_consensus.cc:1081] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:35.604869 10585 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:12:35.605460 10585 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:35.605818 10585 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:35.615882 10585 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.616245 10581 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.633548 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:35.634301 10433 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:35.634634 10433 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:35.636133 10433 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.636952 10585 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:35.637439 10433 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:35.639237 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:35.641099 10431 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:35.641538 10431 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:35.642050 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:35.642696 10508 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:35.642349 10431 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.643675 10431 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:35.640949 10581 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:35.640054 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:35.647516 10511 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:35.661720 10511 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:35.672385 10433 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:35.667400 10431 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:35.683940 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:35.684635 17199 ts_tablet_manager.cc:933] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:12:35.684743 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:35.685030 17160 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:35.685700 17160 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:35.688623 10508 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:35.687086 16956 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:35.696991 16956 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:35.713588 17199 tablet_copy_client.cc:323] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:12:35.719496 10447 tablet_copy_service.cc:140] P 99d42e7288f145f1bd5af212fd062995: Received BeginTabletCopySession request for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:54450)
I20260709 12:12:35.719635 16956 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.720237 10447 tablet_copy_service.cc:161] P 99d42e7288f145f1bd5af212fd062995: Beginning new tablet copy session on tablet 4fa6e3eaefd14aff89099fd64eeb1a1d from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:54450: session id = 52bae2d2caf94d97b1049ecd4f82a2df-4fa6e3eaefd14aff89099fd64eeb1a1d
I20260709 12:12:35.697185 17160 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:35.769352 10328 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 16 to 17, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 17 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:35.792618 10447 tablet_copy_source_session.cc:215] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:35.801177 17199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:35.824781 10335 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 16 to 17, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:35.826840 10335 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:35.836591 10587 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:35.894207 11215 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:35.895974 17197 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:35.907673 10508 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:12:35.908635 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 147a37ae746d4efb84fdc2f6d4d5981e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:35.911309 17159 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:35.935842 10431 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:35.944024 17197 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:35.948511 10433 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:35.959728 17199 tablet_copy_client.cc:806] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:35.960568 17199 tablet_copy_client.cc:670] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:35.967693 16956 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:35.977511 10433 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:35.983541 17197 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:12:36.036360 10508 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:36.032186 16956 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:36.088289 10587 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:12:36.107837 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106446us
I20260709 12:12:36.122480 17199 tablet_copy_client.cc:538] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:36.109035 17130 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:12:36.132118 10508 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:36.139585 10333 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 14, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:36.144767 17155 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:36.148168 10431 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:36.166554 17160 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:12:36.202735 10508 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:12:36.216609 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62606.4us
I20260709 12:12:36.224213 10433 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:12:36.222370 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51794.9us
I20260709 12:12:36.255153 16956 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:36.260999 10433 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:36.267805 10506 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:36.312187 17159 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:36.330447 17199 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:12:36.355769 17218 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:12:36.356173 10587 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:36.392074 10433 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:12:36.397464 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8daad82d9f346a295fe1cf354383acc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:36.426200 17227 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:36.450385 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62109.9us
I20260709 12:12:36.453338 17223 ts_tablet_manager.cc:933] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:36.490903 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87088.5us
W20260709 12:12:36.502344 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 76870.7us
I20260709 12:12:36.508397  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:36.534478 17088 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:36.716259 17232 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:12:36.759620 17218 log.cc:927] Time spent T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:12:36.850695 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 358575us
I20260709 12:12:36.852322 17228 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Committing config change with OpId 19.10: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:36.855871 17223 tablet_copy_client.cc:323] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
W20260709 12:12:36.843642 17199 log.cc:927] Time spent T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.093s	user 0.000s	sys 0.011s
W20260709 12:12:36.906423 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 204687us
I20260709 12:12:36.956992 10432 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Committing config change with OpId 19.10: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:37.020079 17250 ts_tablet_manager.cc:933] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:37.030664 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49423.8us
W20260709 12:12:37.032313 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 118973us
I20260709 12:12:37.077556 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 147a37ae746d4efb84fdc2f6d4d5981e from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:12:37.078625 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 147a37ae746d4efb84fdc2f6d4d5981e from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-147a37ae746d4efb84fdc2f6d4d5981e
I20260709 12:12:37.078313 10328 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:37.112860 10584 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:37.115702 10578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:37.116412 10578 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:37.183733 10433 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:37.184592 10433 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:37.163308 17250 tablet_copy_client.cc:323] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:12:37.188513 17199 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:37.189851 17199 tablet_bootstrap.cc:492] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:37.190232 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:37.190698 17199 ts_tablet_manager.cc:1403] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.861s	user 0.034s	sys 0.031s
I20260709 12:12:37.190872 10506 raft_consensus.cc:606] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:37.195477 10522 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet e8daad82d9f346a295fe1cf354383acc from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:12:37.202695 10581 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Committing config change with OpId 19.10: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:37.207669 10597 tablet_copy_source_session.cc:215] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:37.242511  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:12:37.245086  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:37.237226 10522 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet e8daad82d9f346a295fe1cf354383acc from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-e8daad82d9f346a295fe1cf354383acc
I20260709 12:12:37.258832 10506 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:37.260076 17223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 147a37ae746d4efb84fdc2f6d4d5981e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:37.275794 17247 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:37.276331 10522 tablet_copy_source_session.cc:215] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:37.281790 17250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8daad82d9f346a295fe1cf354383acc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:37.291481 10553 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7919194a8a8542559be104753f043851. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:37.294667 10431 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:37.298030 17217 raft_consensus.cc:993] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:37.298943 17253 raft_consensus.cc:1081] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:37.302466 10584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:37.303172 10584 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:37.304342 10584 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:37.308395 17247 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:12:37.346097 17199 raft_consensus.cc:359] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:37.364404 17199 raft_consensus.cc:740] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:37.415040 17199 consensus_queue.cc:260] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:37.425999 11220 raft_consensus.cc:1217] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Deduplicated request from leader. Original: 12.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 12:12:37.428867  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:37.433228 17248 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:37.440495 10432 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:37.449270 17199 ts_tablet_manager.cc:1434] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.222s	user 0.009s	sys 0.000s
I20260709 12:12:37.457051 10447 tablet_copy_service.cc:342] P 99d42e7288f145f1bd5af212fd062995: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4fa6e3eaefd14aff89099fd64eeb1a1d received from {username='slave'} at 127.0.0.1:54450
I20260709 12:12:37.457634 10447 tablet_copy_service.cc:434] P 99d42e7288f145f1bd5af212fd062995: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4fa6e3eaefd14aff89099fd64eeb1a1d on tablet 4fa6e3eaefd14aff89099fd64eeb1a1d with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:37.473865 10584 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.476939 10513 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:37.507036 10584 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:37.510366 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:37.511107 10511 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:12:37.511446 10511 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:37.509497 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:37.555613 10433 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:37.605201 17233 raft_consensus.cc:993] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:37.606002 17225 raft_consensus.cc:1081] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:37.560328 10511 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.625907 10433 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:12:37.645633 10432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48986
I20260709 12:12:37.647231 10432 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:37.647684 10432 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:37.642596 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:37.640355 17223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57892 (request call id 881) took 1292 ms. Trace:
I20260709 12:12:37.657186 17223 rpcz_store.cc:276] 0709 12:12:36.347705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.347900 (+   195us) service_pool.cc:224] Handling call
0709 12:12:36.495939 (+148039us) ts_tablet_manager.cc:934] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
0709 12:12:37.256329 (+760390us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:37.442048 (+185719us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:37.442229 (+   181us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:37.639440 (+197211us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:37.639833 (+   393us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:37.640175 (+   342us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:37.640237 (+    62us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":70732,"tablet-copy.queue_time_us":48342,"thread_start_us":288716,"threads_started":2}
I20260709 12:12:37.665215 17223 tablet_copy_client.cc:806] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:37.665917 17223 tablet_copy_client.cc:670] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:37.668726 10432 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.673381 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:37.674949 10578 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:37.675197 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:12:37.676223 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 72332.9us
I20260709 12:12:37.676421 10506 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:37.670974 10432 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:37.677377 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 67666.5us
I20260709 12:12:37.675343 10578 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:37.680140 10578 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.688746 10578 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:37.691176 10511 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:37.677731 17247 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:37.698628 17247 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:37.699730 17247 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.702346 17223 tablet_copy_client.cc:538] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:37.730419 17254 raft_consensus.cc:993] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:37.693661 17250 tablet_copy_client.cc:806] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:37.758322 10506 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:37.756181 17250 tablet_copy_client.cc:670] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:37.767486 10509 raft_consensus.cc:1240] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:37.771685 17199 ts_tablet_manager.cc:933] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:37.759588 17252 raft_consensus.cc:1081] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:37.772665 10578 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:37.776341 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:37.778450 17241 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:37.779003 17241 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:37.779944 17241 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.793114 10511 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:12:37.805213 17250 tablet_copy_client.cc:538] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:37.849795 10511 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:37.851016 10511 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:37.851430 10511 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:37.870406 10511 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.878741 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:37.879515 10432 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:37.879864 10432 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:37.880756 10432 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:37.882062 10432 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:37.895910 10511 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:37.898725 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.907187 17199 tablet_copy_client.cc:323] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:12:37.921320 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 7919194a8a8542559be104753f043851 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:12:37.922017 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 7919194a8a8542559be104753f043851 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-7919194a8a8542559be104753f043851
I20260709 12:12:37.931901 10597 tablet_copy_source_session.cc:215] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:37.937283 17223 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:12:37.940265 10333 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:37.948834 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:37.949550 10578 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:37.959960 10432 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:37.961462 10479 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:37.980515 10578 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:37.994469 17250 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:12:38.004022 17253 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:38.013161 17253 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:38.014292 17253 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:38.019320 17104 raft_consensus.cc:1064] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:38.023531 17199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7919194a8a8542559be104753f043851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:38.022498 17104 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:38.083659 10335 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 10 to 11, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 11 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:38.098861 11220 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:12:38.100586 17254 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:38.101670 10579 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:12:38.103056 10511 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:12:38.103996 17241 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:38.108157 17231 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:38.180640 10434 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:12:38.183189 10328 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 16, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:38.188906 10333 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 20 to 21, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:38.207773 17249 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:38.212353 10581 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:38.224593 11220 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:38.195039 17248 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:12:38.238935 17272 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:12:38.278383 10581 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:38.278463 10513 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:38.297523 10509 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:12:38.300853 17159 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:38.298087 17254 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:12:38.322167 10328 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:38.429018 10581 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:38.430807 17227 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:38.447563 10431 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:38.481076 17253 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:38.493597 10432 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:38.509009 11214 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:12:38.515633 17199 tablet_copy_client.cc:806] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:38.517409  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:38.519234 10509 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:38.603940 17199 tablet_copy_client.cc:670] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
W20260709 12:12:38.647106 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 144104us
W20260709 12:12:38.652971 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 142315us
W20260709 12:12:38.655539 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50826.4us
W20260709 12:12:38.679441 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 80708.9us
W20260709 12:12:38.674232 17268 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:12:38.674300 17254 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:12:38.833194 17305 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:12:38.899694 17286 log.cc:927] Time spent T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:12:38.913874 17199 tablet_copy_client.cc:538] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:38.933836 17250 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=2 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:38.940222 17250 tablet_bootstrap.cc:492] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:38.941210 17250 ts_tablet_manager.cc:1403] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.947s	user 0.054s	sys 0.023s
W20260709 12:12:38.955811 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 143036us
W20260709 12:12:39.013837 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 414494us
I20260709 12:12:39.020533 10431 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
W20260709 12:12:39.026638 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 357063us
I20260709 12:12:39.052517 10431 raft_consensus.cc:606] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:39.094002 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 294408us
I20260709 12:12:39.112628 17250 raft_consensus.cc:359] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:39.114096 17250 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:39.119915 17250 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:39.266474 10506 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:12:39.281273 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 91776.7us
I20260709 12:12:39.304795 17250 ts_tablet_manager.cc:1434] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.363s	user 0.011s	sys 0.001s
W20260709 12:12:39.350968 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63555us
W20260709 12:12:39.389460 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 206038us
W20260709 12:12:39.414844 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 102485us
I20260709 12:12:39.393975 17254 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:12:39.422186 17199 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:12:39.422034 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:39.422863 10579 raft_consensus.cc:606] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:39.425309 17317 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:39.432996 11220 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:39.455819 10522 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-e8daad82d9f346a295fe1cf354383acc received from {username='slave'} at 127.0.0.1:38166
I20260709 12:12:39.456034 17252 raft_consensus.cc:993] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:39.456532 10522 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-e8daad82d9f346a295fe1cf354383acc on tablet e8daad82d9f346a295fe1cf354383acc with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:39.456741 17252 raft_consensus.cc:1081] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:39.494072 10506 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:39.494910 10506 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:39.495347 10506 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.526683 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:39.527599 10505 raft_consensus.cc:606] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:39.533604 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:39.534696 10505 raft_consensus.cc:606] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:39.540017 10506 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:39.604561 10513 raft_consensus.cc:2955] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:39.607292  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 12:12:39.611382 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 43924.4us
I20260709 12:12:39.613102 10506 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:39.615826 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44229.7us
W20260709 12:12:39.616595 17285 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:12:39.643788  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:39.645057 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:39.645759 10579 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.671770 17291 raft_consensus.cc:993] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:39.672590 17324 raft_consensus.cc:1081] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:39.685824 10582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:39.686579 10582 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:39.687026 10582 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:39.690045 10579 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:39.703951 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:39.704895 10437 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:39.705304 10437 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:39.690462 10335 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:39.719163  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:39.718083 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:39.727243 10437 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:39.732532 10437 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.742769 17312 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:39.754750 17313 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:39.755613 17313 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:39.771809 17312 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:39.773051 17312 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:39.775506 17227 raft_consensus.cc:993] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
W20260709 12:12:39.818300 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47999.2us
W20260709 12:12:39.835173 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 93926.1us
W20260709 12:12:39.839811 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47047.2us
I20260709 12:12:39.845686 10582 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:39.847689 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 107765us
I20260709 12:12:39.853602 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:39.850762 17223 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:12:39.864408 10437 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:39.869354 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:39.869779 10582 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:39.876784 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:39.877477 10508 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:39.877805 10508 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:39.871609 17293 raft_consensus.cc:1081] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:39.894471 10508 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:39.894508 10553 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:39.904050 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:39.868884 17313 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:39.910301 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:39.932829 17223 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:39.934077 17223 tablet_bootstrap.cc:492] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:39.922688 17225 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:39.934929 17223 ts_tablet_manager.cc:1403] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 1.998s	user 0.071s	sys 0.027s
I20260709 12:12:39.940294 10429 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:39.953210 17290 raft_consensus.cc:993] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:39.953862 17290 raft_consensus.cc:1081] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:39.961457 10508 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:39.970347 10429 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:39.974346 10513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:39.975057 10513 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:39.975468 10513 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:39.940460 17317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51568 (request call id 3) took 1435 ms. Trace:
I20260709 12:12:39.979568 17317 rpcz_store.cc:276] 0709 12:12:38.490896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.491076 (+   180us) service_pool.cc:224] Handling call
0709 12:12:38.501378 (+ 10302us) log.cc:844] Serialized 487 byte log entry
0709 12:12:38.646768 (+145390us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b180045e980. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c92a38b0b 00007f2c92abdb63 00007f2c92ae5752 00007f2c92add9fa
0709 12:12:38.831879 (+185111us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002e4120. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:12:38.916478 (+ 84599us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b2c00000f48. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e7a0a 00007f2c913e7693 00007f2c913e7631 00007f2c92b1bb52 00007f2c92b1bcf5 00007f2c92a58691 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69
0709 12:12:39.910243 (+993765us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":185292,"raft.queue_time_us":151674,"raft.run_cpu_time_us":5096,"raft.run_wall_time_us":250776,"spinlock_wait_cycles":483200512}
I20260709 12:12:39.981267 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:39.990633 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:40.000118 10508 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:40.001205 10429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
W20260709 12:12:40.003165 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 108757us
I20260709 12:12:40.011235 10429 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:40.012903 10429 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:40.020205 10513 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:40.021895 10429 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:40.026136 10513 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:40.011976 17290 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:40.033357 17290 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:40.034466 17290 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:40.035044 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:40.035748 10513 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:40.036039 10513 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:40.036957 10513 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:40.037914 10513 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:40.043612 10429 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:40.049306 10513 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:40.051087 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:40.050050 17223 raft_consensus.cc:359] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:40.055497 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:40.056173 10579 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:40.058152 11220 raft_consensus.cc:1217] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Deduplicated request from leader. Original: 19.9->[19.10-19.10]   Dedup: 19.10->[]
I20260709 12:12:40.055567 17301 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:40.064134 17301 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:40.065052 17301 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:40.080070 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:40.080870 11212 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:40.081435 17199 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:40.082515 17199 tablet_bootstrap.cc:492] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:40.083413 17199 ts_tablet_manager.cc:1403] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.716s	user 0.053s	sys 0.010s
I20260709 12:12:40.086795 17199 raft_consensus.cc:359] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:40.087939 17199 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:40.088577 17199 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:40.112326 10579 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:40.128561 17199 ts_tablet_manager.cc:1434] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.004s
W20260709 12:12:40.134001 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:12:40.133414 11212 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:40.055925 17223 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:40.139106 17223 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:40.142071 17223 ts_tablet_manager.cc:1434] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.207s	user 0.007s	sys 0.001s
I20260709 12:12:40.147950 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:40.148852 10581 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:40.149255 10581 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:40.150285 10581 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:40.151499 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-7919194a8a8542559be104753f043851 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:12:40.156414 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-7919194a8a8542559be104753f043851 on tablet 7919194a8a8542559be104753f043851 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:40.167791 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:40.183104 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-147a37ae746d4efb84fdc2f6d4d5981e received from {username='slave'} at 127.0.0.1:36746
I20260709 12:12:40.183751 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-147a37ae746d4efb84fdc2f6d4d5981e on tablet 147a37ae746d4efb84fdc2f6d4d5981e with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:40.222440 17299 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:40.237680 17299 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:40.238837 17299 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:40.248381 10587 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:40.255764 10581 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:40.260012 11220 raft_consensus.cc:1240] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:40.269435 17321 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:40.279734 10581 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:40.286289 10333 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:40.358418 10437 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:40.365895 17324 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:12:40.374645 10333 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 17 to 18, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:12:40.413897 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 4fa6e3eaefd14aff89099fd64eeb1a1d with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:55454
I20260709 12:12:40.451821 11220 raft_consensus.cc:1217] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 12:12:40.477560 17291 raft_consensus.cc:1064] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:40.480003 10434 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:40.481062 17291 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 8, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:40.484071 17351 tablet_replica.cc:333] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:12:40.488041 17351 raft_consensus.cc:2243] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:40.489095 17351 raft_consensus.cc:2272] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:40.495618 17247 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:40.498219 10333 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 19 to 20, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.523985 17351 ts_tablet_manager.cc:1916] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:40.577189 10581 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:40.587965 17324 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:40.588506 10434 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:40.593044 11214 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:40.595248 17312 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:40.597393 17312 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:40.636596 10513 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:40.678321 17317 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:12:40.694669 10333 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 17 to 18, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:40.712558 11220 raft_consensus.cc:1217] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Deduplicated request from leader. Original: 20.13->[20.14-20.14]   Dedup: 20.14->[]
I20260709 12:12:40.720525 10505 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:40.724699 17311 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Committing config change with OpId 19.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:40.734259 10435 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:40.765303 17351 ts_tablet_manager.cc:1929] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.10
I20260709 12:12:40.765924 17351 log.cc:1199] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/4fa6e3eaefd14aff89099fd64eeb1a1d
I20260709 12:12:40.773319 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 4fa6e3eaefd14aff89099fd64eeb1a1d (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:40.745051 17225 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:12:40.780038 10579 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:12:40.768184 17355 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:12:40.802115 10579 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:40.804041 17264 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:12:40.823750 17278 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:12:40.931480 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 100299us
I20260709 12:12:40.960129 10328 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:40.993165 10579 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.027107 17290 raft_consensus.cc:1064] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:41.041337 11212 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:12:41.086786 17290 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:41.049096 11212 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.142357 10508 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:41.172739 17311 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:12:41.194518 10431 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:41.196064 10513 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:41.202607 10435 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:41.203022 11214 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:41.204634 17247 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:41.215888 17290 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:41.228935 17361 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:41.233222 11214 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:41.273342 17323 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:41.313611 10434 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.310088 17321 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.342795  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:41.346316 17290 raft_consensus.cc:1064] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:41.363801 17361 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:12:41.391834 11212 raft_consensus.cc:2955] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.398469 17290 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:41.442209 17371 log.cc:927] Time spent T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:12:41.557027 11220 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.594940 10435 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.476776 17289 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:41.645756 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e8daad82d9f346a295fe1cf354383acc with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:12:41.659209 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43502.8us
W20260709 12:12:41.677255 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 103510us
W20260709 12:12:41.680397 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 110760us
I20260709 12:12:41.714453 10435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:41.716399 10435 raft_consensus.cc:606] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:41.739831 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:41.740636 10579 raft_consensus.cc:606] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:41.725955 10335 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:41.777261 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 79515.9us
W20260709 12:12:41.782927 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84516us
I20260709 12:12:41.785588 10513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:41.786243 10513 raft_consensus.cc:606] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:41.786645 10513 raft_consensus.cc:630] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Rejecting request to transfer leadership because tablet server 33d77e3063684586a737bccf358786d8 is not a voter in the active config
W20260709 12:12:41.808715 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 111319us
I20260709 12:12:41.843575 11220 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:12:41.916702 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 92749.3us
I20260709 12:12:41.911880 17288 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:12:41.960119 10513 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.021476 10509 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:12:42.166733 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 61695.8us
W20260709 12:12:42.179697 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 261596us
I20260709 12:12:42.188624  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:12:42.261152 17288 raft_consensus.cc:993] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:42.278257 17288 raft_consensus.cc:1081] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
W20260709 12:12:42.287410 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 110770us
I20260709 12:12:42.295415 17396 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:12:42.316048  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:42.323711 10333 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 11 to 12, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.383584  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:42.387948 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet e8daad82d9f346a295fe1cf354383acc with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:55454
I20260709 12:12:42.402884 10438 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:12:42.452226 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 71885us
I20260709 12:12:42.494012 10438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48986
I20260709 12:12:42.494895 10438 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:42.495390 10438 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.502265 17396 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:42.503340 17396 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:42.511060 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:42.513355 17396 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:42.513731 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:42.535604 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:42.551554 17390 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:42.564052 17390 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:42.568542 17363 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:42.571532 17399 tablet_replica.cc:333] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:12:42.566890 17390 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:42.575168 10553 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:42.587141 17288 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:42.592772 17360 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.600401 11213 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.607514 10508 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.638437 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:42.640974 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:42.664522 17394 raft_consensus.cc:993] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:42.665251 17390 raft_consensus.cc:1081] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:42.651578 10438 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:42.671728 17399 raft_consensus.cc:2243] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:42.673748 17399 raft_consensus.cc:2272] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:42.685511 10584 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:42.708245 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:42.671828 10438 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:42.713595 17399 ts_tablet_manager.cc:1916] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:42.714952 10332 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 14 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.734275 17389 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:42.740931 10435 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:42.745880 10508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:42.746765 10508 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:42.747192 10508 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:42.750777 17396 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:42.753248 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:42.752032 17396 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:42.755331 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:42.757707 10511 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.756346 17396 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:42.785876 10438 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:42.787732 10508 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:42.809247 17396 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:42.811219 17396 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.814842 17288 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:42.837733 11213 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:42.841265 17288 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:12:42.852573 10511 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:42.871934 10513 raft_consensus.cc:1240] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:42.874915 17382 consensus_queue.cc:1059] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:42.876616 17358 raft_consensus.cc:3055] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:42.877038 17358 raft_consensus.cc:740] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:42.878047 17358 consensus_queue.cc:260] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:42.880077 17358 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:42.883508 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:42.884186 11214 raft_consensus.cc:3060] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:42.882546 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:42.878702 10508 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:42.922632 11214 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:42.924125 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:42.942734 10435 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:42.946583 17394 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:42.947244 17394 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:42.948348 17394 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:42.955740 10434 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.968220 10479 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:42.970453 17388 raft_consensus.cc:2804] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:42.971041 17388 raft_consensus.cc:697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:42.972008 17388 consensus_queue.cc:237] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:42.973356 17288 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.983800 10429 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:42.993399 10582 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:42.993417 11213 raft_consensus.cc:2955] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:43.001056 11214 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:12:43.005302 17290 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:12:43.015149 10437 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:12:43.040571 10584 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:43.057897 17399 ts_tablet_manager.cc:1929] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.11
I20260709 12:12:43.058565 17399 log.cc:1199] T e8daad82d9f346a295fe1cf354383acc P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/e8daad82d9f346a295fe1cf354383acc
I20260709 12:12:43.070503 10328 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 17, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 17 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:43.077469 17395 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:43.090554 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet e8daad82d9f346a295fe1cf354383acc (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:43.091627 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7919194a8a8542559be104753f043851 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:43.101569 17357 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:12:43.110553 11214 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:43.122184 10335 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 12 to 13, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:43.138335 10333 catalog_manager.cc:5697] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 20 to 21, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:43.148175 10437 raft_consensus.cc:2955] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:43.156708 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet 7919194a8a8542559be104753f043851 with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:43.171801 10335 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 15 to 16, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:43.188766 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 147a37ae746d4efb84fdc2f6d4d5981e with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:43.212392 17406 tablet_replica.cc:333] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:12:43.216746 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet 147a37ae746d4efb84fdc2f6d4d5981e with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:43.218209 17406 raft_consensus.cc:2243] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:43.223891 17406 raft_consensus.cc:2272] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:43.229352 17406 ts_tablet_manager.cc:1916] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:43.375926 17406 ts_tablet_manager.cc:1929] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 12:12:43.383297 17406 log.cc:1199] T 7919194a8a8542559be104753f043851 P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/7919194a8a8542559be104753f043851
I20260709 12:12:43.393498 17406 tablet_replica.cc:333] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:12:43.395009 17406 raft_consensus.cc:2243] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:43.396020 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet 7919194a8a8542559be104753f043851 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:43.397362 17406 raft_consensus.cc:2272] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:43.397440 10511 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:43.401928 17394 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:43.451215 10581 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:43.453574 17382 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:43.490618 11214 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:43.534976 10434 raft_consensus.cc:1275] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:43.555049 17311 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:43.575511 17264 consensus_queue.cc:1048] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:43.579866 17406 ts_tablet_manager.cc:1916] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:43.740587 17406 ts_tablet_manager.cc:1929] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.15
I20260709 12:12:43.754382 17406 log.cc:1199] T 147a37ae746d4efb84fdc2f6d4d5981e P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/147a37ae746d4efb84fdc2f6d4d5981e
I20260709 12:12:43.768971 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet 147a37ae746d4efb84fdc2f6d4d5981e (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:44.171794  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:44.528216 10435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:44.529211 10435 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:44.533738 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:44.534441 10581 raft_consensus.cc:606] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:44.539026 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:44.539729 10511 raft_consensus.cc:606] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:44.543062 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:44.543742 10511 raft_consensus.cc:606] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:44.546708 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:44.547355 10511 raft_consensus.cc:606] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:44.548573  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:44.550913  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:44.551808  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:44.575801 10511 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:44.585747 17409 raft_consensus.cc:993] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:44.586433 17409 raft_consensus.cc:1081] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:44.589161 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:44.589885 10581 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:44.590365 10581 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:44.608819 10578 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:44.610723 17425 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:44.617305 10435 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:44.607234 10581 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:44.620329 17396 raft_consensus.cc:993] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:12:44.620944 17396 raft_consensus.cc:1081] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:12:44.622974 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:44.624735 11221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7919194a8a8542559be104753f043851"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57892
I20260709 12:12:44.625373 11221 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:44.625808 11221 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:44.639582 10479 consensus_peers.cc:597] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ecefe266eea48c4af129b425f391fe9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:44.648722 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:44.649626 10437 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:44.624112 10511 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:44.646203 10581 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:44.654857 10511 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:44.648650 17425 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:44.656020 10511 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:44.657683 10511 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:44.660457 17425 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:44.666231 10581 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:44.640856 11221 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:44.706458 10437 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:12:44.714706 10511 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:12:44.750701 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:44.751514 10578 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:44.751859 10578 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:44.753110 10578 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:44.759187 11221 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:44.754501 10578 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:44.760246 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:44.764802 17423 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:44.765756 17423 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:44.769526 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:44.775672 10437 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:44.766825 17423 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:44.792315 10332 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 16 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:44.804579 10578 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:12:44.808001 10437 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:12:44.813196 17424 raft_consensus.cc:993] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:44.813807 17409 raft_consensus.cc:1081] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:44.814607 11177 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:44.815861 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:44.816498 10581 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:44.816898 10581 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:44.808771 10431 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:44.819794 17419 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:44.820806 17419 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:44.823618 17419 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:44.841037 10581 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:44.855406 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:44.856228 10513 raft_consensus.cc:3055] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:44.856572 10513 raft_consensus.cc:740] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:44.861176 10581 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:44.865376 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:44.866119 11221 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:44.857434 10513 consensus_queue.cc:260] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:44.875494 10513 raft_consensus.cc:3060] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:44.943385 10581 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:44.946281 17420 raft_consensus.cc:993] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:44.952801 17425 raft_consensus.cc:1081] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:44.956377 10437 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:44.956959 10437 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:44.957178 10513 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:12:44.957412 10437 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:44.958101 17421 raft_consensus.cc:993] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:44.958923 10552 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:44.959371 11221 raft_consensus.cc:2468] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:12:44.958920 17422 raft_consensus.cc:1081] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:44.961581 17423 raft_consensus.cc:2804] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:44.962077 17423 raft_consensus.cc:697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:44.964272 17423 consensus_queue.cc:237] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:44.970355 10513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:44.970883 10513 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:44.974376 10513 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:45.011842 10437 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:45.042188 10437 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:45.041738 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:45.047564 10507 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:45.047726 10513 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:45.048008 10507 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:45.051271 10507 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:45.052834 10507 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:45.052929 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:45.053635 10431 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:45.053941 10431 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:45.054771 10431 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:45.055981 10431 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:45.075096 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:45.076042 11221 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:45.075722 10507 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:45.091023 10513 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:45.102756 10431 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
W20260709 12:12:45.112087 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 64791us
W20260709 12:12:45.113754 17432 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:12:45.119413 11221 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:45.124804 10328 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 17 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 18 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:45.120471 10431 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:12:45.128896 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49598.3us
I20260709 12:12:45.131273 10513 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:45.132467 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:45.133356 10578 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:45.143280 10578 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:12:45.148046 10478 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:45.154148 17448 ts_tablet_manager.cc:933] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:45.163858 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:45.165092 17394 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:12:45.170674 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 43660.3us
W20260709 12:12:45.192750 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44466.9us
I20260709 12:12:45.165647 17394 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:45.169628 17409 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:12:45.211627 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116724us
I20260709 12:12:45.214058 17439 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:12:45.231827 17394 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:45.251144 17409 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:45.252399 17409 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:45.270699 17448 tablet_copy_client.cc:323] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
W20260709 12:12:45.323634 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 108251us
I20260709 12:12:45.388883 10431 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:12:45.392397 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 90725.1us
W20260709 12:12:45.419099 17447 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.209s	user 0.004s	sys 0.000s
I20260709 12:12:45.428512 17455 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:12:45.433307 10553 consensus_peers.cc:597] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc98cd5edc9240fba53978ed59191cbd. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:45.439414 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 170715us
I20260709 12:12:45.488930 10522 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 1ecefe266eea48c4af129b425f391fe9 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:12:45.489719 10522 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 1ecefe266eea48c4af129b425f391fe9 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-1ecefe266eea48c4af129b425f391fe9
I20260709 12:12:45.504546 10511 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:45.537662 10522 tablet_copy_source_session.cc:215] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:12:45.592442 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 198090us
I20260709 12:12:45.585978 17463 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:12:45.633944 10511 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:12:45.639359 11213 raft_consensus.cc:1275] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:12:45.645483 17442 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:12:45.647842 17438 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 12:12:45.653409 10431 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:45.671959 10511 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:45.681694 17433 consensus_queue.cc:1048] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:12:45.669940 17467 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:45.699136 10578 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:45.725677 10581 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:12:45.738180  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:45.728816 17449 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:12:45.760016 10333 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:45.761951 10333 catalog_manager.cc:5697] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 20 to 21, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 21 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:45.774739 17448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ecefe266eea48c4af129b425f391fe9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:45.809641 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 72302.2us
I20260709 12:12:45.811820 17419 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:45.808779 10335 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 20 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:45.826871 17462 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:45.839745 17477 ts_tablet_manager.cc:933] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:45.925048 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44285.8us
I20260709 12:12:45.932161 10437 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:45.973356 17477 tablet_copy_client.cc:323] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:12:45.978811 17264 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:45.981197 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45913.9us
I20260709 12:12:45.998369 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:45.999341 10511 raft_consensus.cc:606] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:46.013985 11221 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:46.060793 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet dc98cd5edc9240fba53978ed59191cbd from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:12:46.061192 17467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:50898 (request call id 2) took 1123 ms. Trace:
I20260709 12:12:46.061656 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet dc98cd5edc9240fba53978ed59191cbd from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-dc98cd5edc9240fba53978ed59191cbd
I20260709 12:12:46.068787 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:46.069571 10511 raft_consensus.cc:606] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:46.071466 17454 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:46.061771 17467 rpcz_store.cc:276] 0709 12:12:44.937748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.937909 (+   161us) service_pool.cc:224] Handling call
0709 12:12:44.955354 (+ 17445us) log.cc:844] Serialized 492 byte log entry
0709 12:12:46.061083 (+1105729us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40447,"raft.queue_time_us":111789,"raft.run_cpu_time_us":4428,"raft.run_wall_time_us":143863,"thread_start_us":164658,"threads_started":2,"wal-append.queue_time_us":62141}
I20260709 12:12:46.113647 10437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:46.129963 10597 tablet_copy_source_session.cc:215] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:46.131523 10436 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:46.156049 10437 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:46.156809 10437 raft_consensus.cc:614] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:46.157428 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 137221us
W20260709 12:12:46.168932 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 101295us
I20260709 12:12:46.191676 10584 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:46.223660 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:46.224400 10507 raft_consensus.cc:606] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
W20260709 12:12:46.241505 17490 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:12:46.260914 10508 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:12:46.304651 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:46.307927 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 63382.2us
I20260709 12:12:46.319864 10332 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 13 to 14, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:46.366528  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:46.369585  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:12:46.427698 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47843.4us
I20260709 12:12:46.478457  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:46.516674 17477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc98cd5edc9240fba53978ed59191cbd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:46.651922 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47413.2us
W20260709 12:12:46.670842 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44694.7us
I20260709 12:12:46.721714 10435 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:46.745065 17449 raft_consensus.cc:993] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:46.746094 17491 raft_consensus.cc:1081] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
W20260709 12:12:46.757042 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 43755.3us
I20260709 12:12:46.773233 10584 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:46.773897 10584 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:46.774297 10584 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:46.786489 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 357707us
I20260709 12:12:46.790369 10508 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:46.822479 17502 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.831934 17502 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:46.834208 17502 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:46.823355 17435 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:12:46.794432 17459 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:12:46.841920 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 69970.1us
I20260709 12:12:46.854700 10584 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:46.858203 17450 raft_consensus.cc:993] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:46.825730 17448 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57904 (request call id 1025) took 1727 ms. Trace:
I20260709 12:12:46.867081 17492 raft_consensus.cc:993] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:46.888563 10584 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:12:46.891089 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 50082.2us
W20260709 12:12:46.905288 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 98888.7us
I20260709 12:12:46.906773 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:46.910111 10506 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:46.912590 17425 raft_consensus.cc:1081] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:46.910708 10506 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:46.928242 17492 raft_consensus.cc:1081] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:46.910063 17466 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:12:46.940255 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:46.948987 10585 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:46.949694 10585 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:46.950090 10585 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:46.954495 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:46.957787 10438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:46.958565 10438 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:46.958935 10438 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:46.943318 10506 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:46.978502 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:46.992851 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:46.993414 10506 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:46.997084 10437 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:46.998342 10585 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:47.008198 17493 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:46.887270 17448 rpcz_store.cc:276] 0709 12:12:45.095648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.095870 (+   222us) service_pool.cc:224] Handling call
0709 12:12:45.154777 (+ 58907us) ts_tablet_manager.cc:934] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:12:45.726244 (+571467us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:46.318492 (+592248us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:46.318702 (+   210us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:46.786191 (+467489us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:46.786556 (+   365us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:46.804719 (+ 18163us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:46.807860 (+  3141us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":55678,"tablet-copy.queue_time_us":40669,"thread_start_us":83357,"threads_started":2}
I20260709 12:12:47.011772 17448 tablet_copy_client.cc:806] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:47.012477 17448 tablet_copy_client.cc:670] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:47.024987 17477 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57892 (request call id 980) took 1369 ms. Trace:
I20260709 12:12:47.024852 17423 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.036532 10513 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.041091 17448 tablet_copy_client.cc:538] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:47.025588 17477 rpcz_store.cc:276] 0709 12:12:45.655582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.655729 (+   147us) service_pool.cc:224] Handling call
0709 12:12:45.840151 (+184422us) ts_tablet_manager.cc:934] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
0709 12:12:46.370680 (+530529us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:12:46.831328 (+460648us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:46.831484 (+   156us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:12:46.996508 (+165024us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:12:47.024348 (+ 27840us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:47.024840 (+   492us) ts_tablet_manager.cc:989] Replica registered.
0709 12:12:47.024910 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":49337,"tablet-copy.queue_time_us":132907,"thread_start_us":143767,"threads_started":2}
I20260709 12:12:47.050777 17477 tablet_copy_client.cc:806] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:47.051839 17477 tablet_copy_client.cc:670] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:47.053194 10506 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:47.057329 10585 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:47.060264 10437 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:47.081259 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:47.082670 10438 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:47.068279 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:12:47.063808 10553 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 955c0a21e1e04f51bbbc7c06b5d611f5. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:47.091730 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 170031us
I20260709 12:12:47.094378 10508 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:47.091224 17477 tablet_copy_client.cc:538] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:47.094848 10508 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:47.095795 10508 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:47.097236 10508 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.094249 10335 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.113575 17433 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:47.114598 17433 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:47.115731 17433 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:47.130523 10584 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:47.130920 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:47.131676 10429 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.139344 10438 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:47.144132 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:47.146721 11214 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.160610 17448 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:12:47.178960 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:47.237994 10506 raft_consensus.cc:3055] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:47.238444 10506 raft_consensus.cc:740] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:47.182955 17477 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:12:47.203027 10332 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 18 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.212886 17423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:50898 (request call id 3) took 1024 ms. Trace:
I20260709 12:12:47.239506 17423 rpcz_store.cc:276] 0709 12:12:46.188941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.189201 (+   260us) service_pool.cc:224] Handling call
0709 12:12:46.235704 (+ 46503us) log.cc:844] Serialized 492 byte log entry
0709 12:12:46.741766 (+506062us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b180041f5c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:46.745086 (+  3320us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b540035df20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:12:46.873689 (+128603us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b180041f5c0. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:47.212759 (+339070us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15265,"raft.queue_time_us":586,"raft.run_cpu_time_us":2627,"raft.run_wall_time_us":139843,"spinlock_wait_cycles":31885952,"thread_start_us":385636,"threads_started":1,"wal-append.queue_time_us":461233}
I20260709 12:12:47.243650 10506 consensus_queue.cc:260] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:47.231807 10436 raft_consensus.cc:1240] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:47.217809 10429 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:12:47.321659 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50220.8us
I20260709 12:12:47.281992 10508 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:47.323989 10506 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.356606 10511 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:47.370148 10552 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:47.371192 17471 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:47.371789 17471 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:47.327221 11214 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:47.372759 17471 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:47.383671 10506 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:47.384133 17522 ts_tablet_manager.cc:933] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:47.388003 10404 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:47.389322 17422 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:47.389828 17422 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:47.390823 17422 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:47.477877 10332 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 17 to 18, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 18 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:47.478294 10436 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.474941 17522 tablet_copy_client.cc:323] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:12:47.546789 10508 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:12:47.551467 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 65460.3us
I20260709 12:12:47.575699 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 955c0a21e1e04f51bbbc7c06b5d611f5 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:12:47.582295 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 955c0a21e1e04f51bbbc7c06b5d611f5 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-955c0a21e1e04f51bbbc7c06b5d611f5
I20260709 12:12:47.604945 10437 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:47.610085 17425 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:47.587456 17423 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 12:12:47.614813 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59573.8us
W20260709 12:12:47.645871 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 79835.7us
W20260709 12:12:47.691834 10479 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c61f4bfa2db8497dbf7e0446e2ab707d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.702394 10437 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:47.708546 10328 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 17, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 17 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.714418 10597 tablet_copy_source_session.cc:215] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:12:47.720140 17522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 955c0a21e1e04f51bbbc7c06b5d611f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:47.742702 17424 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.747457 10437 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.801214 10583 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:47.756539 17455 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:12:47.759838 10332 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 16 to 17, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:47.829552 17510 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:47.851091 11213 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:47.855701 17394 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:47.877794 10506 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:47.879599 17441 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:12:47.893177 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67232.7us
W20260709 12:12:47.907594 10553 consensus_peers.cc:597] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.908744 10583 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.927870 10437 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:47.961501 10508 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:47.972725 10332 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:47.936321 17530 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:12:47.969014 17527 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:12:48.052258 17522 tablet_copy_client.cc:806] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:48.095290 17522 tablet_copy_client.cc:670] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
W20260709 12:12:48.092312 17543 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.121s	user 0.001s	sys 0.001s
I20260709 12:12:48.083637  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:48.115288 17477 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:48.130842 17477 tablet_bootstrap.cc:492] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:48.131835 17477 ts_tablet_manager.cc:1403] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.949s	user 0.069s	sys 0.033s
W20260709 12:12:48.190629 17448 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:12:48.194419 17544 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.075s	user 0.002s	sys 0.004s
W20260709 12:12:48.299373 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 77233.4us
W20260709 12:12:48.415709 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 66924.1us
W20260709 12:12:48.439283 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 110554us
I20260709 12:12:48.444977 17477 raft_consensus.cc:359] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:48.446167 17477 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:48.447091 17477 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:12:48.475275 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 133915us
W20260709 12:12:48.475898 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 98080.2us
W20260709 12:12:48.486161 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 70722.9us
W20260709 12:12:48.487246 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66840us
I20260709 12:12:48.504319 17522 tablet_copy_client.cc:538] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:12:48.567530 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53624.7us
I20260709 12:12:48.594997 17566 ts_tablet_manager.cc:933] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:48.596560 17477 ts_tablet_manager.cc:1434] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.464s	user 0.007s	sys 0.019s
I20260709 12:12:48.609383 17448 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:48.610630 17448 tablet_bootstrap.cc:492] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
W20260709 12:12:48.631299 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 118965us
I20260709 12:12:48.663084 17448 ts_tablet_manager.cc:1403] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 1.503s	user 0.062s	sys 0.015s
W20260709 12:12:48.679929 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 171273us
I20260709 12:12:48.702909 17448 raft_consensus.cc:359] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:48.736400 17448 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:48.737457 17448 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:48.740757 17448 ts_tablet_manager.cc:1434] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.000s
I20260709 12:12:48.721340 17522 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:12:48.746598 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50653.9us
I20260709 12:12:48.736328 17566 tablet_copy_client.cc:323] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:12:48.796931 11213 raft_consensus.cc:1217] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 12:12:48.817196 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-dc98cd5edc9240fba53978ed59191cbd received from {username='slave'} at 127.0.0.1:36746
I20260709 12:12:48.818480 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-dc98cd5edc9240fba53978ed59191cbd on tablet dc98cd5edc9240fba53978ed59191cbd with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:48.847010 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet c61f4bfa2db8497dbf7e0446e2ab707d from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:12:48.847774 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet c61f4bfa2db8497dbf7e0446e2ab707d from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-c61f4bfa2db8497dbf7e0446e2ab707d
I20260709 12:12:48.849476 10522 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-1ecefe266eea48c4af129b425f391fe9 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:12:48.849934 10522 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-1ecefe266eea48c4af129b425f391fe9 on tablet 1ecefe266eea48c4af129b425f391fe9 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:48.865005 10523 tablet_copy_source_session.cc:215] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:12:48.877369 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46311.6us
I20260709 12:12:48.888275 17566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c61f4bfa2db8497dbf7e0446e2ab707d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:48.892812 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71069.3us
I20260709 12:12:48.914775 10438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:48.915630 10438 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:48.950713 10583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:48.976954 10583 raft_consensus.cc:606] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:48.923146 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 112140us
I20260709 12:12:48.941411 17566 tablet_copy_client.cc:806] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:12:48.979185 17566 tablet_copy_client.cc:670] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:12:48.983556 17566 tablet_copy_client.cc:538] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:12:48.998142 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:48.998818 10581 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:49.006346 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:49.007004 10581 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:49.011277 10581 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:49.012460 10581 raft_consensus.cc:606] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:49.021970  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:49.024487  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:49.026296 17566 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:12:49.028051  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:49.148002 17527 raft_consensus.cc:993] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:49.148700 17459 raft_consensus.cc:1081] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:49.152980 10583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:12:49.153581 10583 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:49.153931 10583 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:49.202749 10583 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:49.210582 10583 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:49.215196 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:49.215837 10437 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:49.216194 10437 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:49.217031 10437 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:49.223528 10437 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:49.238360 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:49.239140 10508 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:49.243957 17441 raft_consensus.cc:993] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:49.244650 17441 raft_consensus.cc:1081] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:49.258731 10437 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:49.260599 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:49.265761 17566 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:49.269244 17560 raft_consensus.cc:993] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:49.273597 17534 raft_consensus.cc:993] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:49.282006 17570 raft_consensus.cc:1081] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:49.294538 17560 raft_consensus.cc:1081] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:49.289904 17471 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:49.305380 17471 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:49.308323 17566 tablet_bootstrap.cc:492] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:49.325778 10505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:49.326565 10505 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:49.326956 10505 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:49.328575 10506 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:49.329253 10506 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:49.329604 10506 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:49.329851 10511 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:49.330403 10511 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:49.330844 10511 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.328689 17566 ts_tablet_manager.cc:1403] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.300s	user 0.035s	sys 0.012s
I20260709 12:12:49.315667 17471 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:49.346226 10508 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:49.354560 10511 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.354104 10506 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.374154 10511 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:49.376457 10506 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:49.392879 11214 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Advancing to term 17
W20260709 12:12:49.426069 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 70409.7us
I20260709 12:12:49.447460 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:49.448282 10583 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:49.448586 10583 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:49.449414 10583 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.463624 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:49.474431 10583 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.491639 17566 raft_consensus.cc:359] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.492798 17566 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:49.493685 17566 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.505309 10581 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:49.505841 10581 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:49.510052 17522 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.132s	user 0.003s	sys 0.000s
I20260709 12:12:49.514544 10511 raft_consensus.cc:1240] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:49.506805 10581 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.519969 17566 ts_tablet_manager.cc:1434] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.178s	user 0.008s	sys 0.002s
I20260709 12:12:49.516332 10505 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:49.520318 10581 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:49.523644 10513 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:49.528105 17576 consensus_queue.cc:1059] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:49.531960 10505 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:49.544884 10583 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:49.552078 10581 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
W20260709 12:12:49.558873 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 134439us
W20260709 12:12:49.624511 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46118.4us
W20260709 12:12:49.628844 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 175491us
I20260709 12:12:49.646435 17522 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=2 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:12:49.648519 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-c61f4bfa2db8497dbf7e0446e2ab707d received from {username='slave'} at 127.0.0.1:38166
I20260709 12:12:49.649084 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-c61f4bfa2db8497dbf7e0446e2ab707d on tablet c61f4bfa2db8497dbf7e0446e2ab707d with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:49.656937 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:49.657645 10583 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:49.657946 10583 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:49.679544 10583 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:49.680600 10583 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:49.712098 10583 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:49.722843 17583 raft_consensus.cc:993] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:49.723536 17583 raft_consensus.cc:1081] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:49.727476 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:49.728173 10434 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.729513 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:49.730186 10438 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:49.714813 17581 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:12:49.733364 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:49.745467 10431 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:49.716540 17522 tablet_bootstrap.cc:492] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:12:49.759910 17522 ts_tablet_manager.cc:1403] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 1.039s	user 0.076s	sys 0.031s
I20260709 12:12:49.785702 10438 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:49.786909 10431 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:49.791718 11221 raft_consensus.cc:1217] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 12:12:49.795478 10435 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48986
I20260709 12:12:49.798805 10435 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:49.799348 10435 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:49.799785 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:49.804104 10434 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:49.804749 10437 raft_consensus.cc:1240] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:49.805855 10479 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:49.810400 17578 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:49.809859 10333 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 16 to 17, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:49.822777 17565 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:49.823503 17565 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:49.824517 17565 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:49.828347 17578 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:49.829442 17578 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.832239 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:49.792632 17590 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:12:49.839926 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 164556us
I20260709 12:12:49.841635 10435 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:49.861550 10435 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:49.791958 17522 raft_consensus.cc:359] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.862728 17522 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:12:49.863730 17522 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:49.843864 17588 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:49.874316 17588 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:49.879925 17588 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:12:49.882082 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46673.3us
W20260709 12:12:49.870882 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 147991us
I20260709 12:12:49.889483 10434 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:49.902787 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:49.903542 10583 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:49.903888 10583 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:49.904838 10583 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:49.906257 10583 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:49.916851 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53275.8us
I20260709 12:12:49.921725 10583 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:12:49.935672 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:49.936466 11221 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:49.946751 17583 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:49.954353 10332 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:49.969094 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44126.6us
I20260709 12:12:49.992398 17522 ts_tablet_manager.cc:1434] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.232s	user 0.018s	sys 0.005s
I20260709 12:12:50.038787 11221 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:12:50.046609 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-955c0a21e1e04f51bbbc7c06b5d611f5 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:12:50.047214 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-955c0a21e1e04f51bbbc7c06b5d611f5 on tablet 955c0a21e1e04f51bbbc7c06b5d611f5 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:50.080621 10508 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:50.087731 17441 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:50.095191 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 142069us
W20260709 12:12:50.133522 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 184119us
W20260709 12:12:50.134897 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 194286us
I20260709 12:12:50.138866 10404 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:50.170848 17394 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:50.171556 17394 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:50.172639 17394 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:50.231324 10435 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:50.240800 17577 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:12:50.249711 10434 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:50.251618 17564 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:12:50.304018 17613 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:12:50.306247 10438 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:50.308182 17578 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:12:50.319702 10583 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:50.322315 10583 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:50.323570 17564 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:50.280290  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:50.337366 11213 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Advancing to term 21
I20260709 12:12:50.344873 17581 raft_consensus.cc:1064] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:50.349614 17564 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:50.353777 17588 raft_consensus.cc:1064] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:50.364820 11213 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:50.368677 10335 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 20 to 21, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 21 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:50.373216 17577 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:12:50.348213 17581 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:50.469514 17613 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:12:50.471800 17609 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:12:50.529157 17614 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:12:50.474495 17588 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:50.593199 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54121.9us
I20260709 12:12:50.616209 11212 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEARNER]: Advancing to term 19
I20260709 12:12:50.634076 11212 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:50.635437 10333 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 20 to 21, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:50.653024 11212 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:12:50.689510 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58577us
I20260709 12:12:50.689702 10582 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:12:50.695250 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46529.8us
I20260709 12:12:50.722882 10505 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:50.723838 10585 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:50.719183 10332 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 19, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:50.745172 17577 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:50.732936 17570 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:12:50.752619 17532 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:50.756260 17588 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:12:50.762681 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 53609.9us
I20260709 12:12:50.763931 10583 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:50.767796 11213 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:50.764622 17595 raft_consensus.cc:1064] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:50.773701 17578 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:12:50.787789 10438 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:12:50.819680 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 98886.9us
I20260709 12:12:50.826881 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:50.827662 10506 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership
W20260709 12:12:50.826777 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53172.6us
I20260709 12:12:50.778321 17595 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:50.823675 17577 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:12:50.860875 17584 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
W20260709 12:12:50.889079 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62449.2us
I20260709 12:12:50.896337 10585 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:50.897182 10585 raft_consensus.cc:606] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:50.934378 11212 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:12:50.937024 17584 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:12:50.946578 10438 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:12:50.957412 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66622.7us
I20260709 12:12:50.963771 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:50.964413 10505 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership
W20260709 12:12:50.965613 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 129899us
W20260709 12:12:50.967097 10322 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 955c0a21e1e04f51bbbc7c06b5d611f5 already in progress
I20260709 12:12:50.980423 17593 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:50.996250 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 81551.9us
I20260709 12:12:51.002507 10579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:51.003222 10579 raft_consensus.cc:606] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:51.061877 10322 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 94766.1us
W20260709 12:12:51.099620 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 58430.7us
I20260709 12:12:51.073628 17423 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:12:51.152869 17628 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.192780 11218 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:51.200345 17576 raft_consensus.cc:993] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:51.201071 17576 raft_consensus.cc:1081] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
W20260709 12:12:51.188892 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 72361.9us
I20260709 12:12:51.192664 10438 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:51.204360 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33776
I20260709 12:12:51.205065 10505 raft_consensus.cc:606] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership
I20260709 12:12:51.211547 10508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:51.212226 10508 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:51.212611 10508 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.216013 11218 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.214862 17565 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:12:51.221443 17606 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:51.238680 10583 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:51.238325 10508 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.241250 10583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:51.241974 10583 raft_consensus.cc:606] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:51.241662 10579 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.257274 17550 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.255139 10507 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.260659 10583 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:12:51.261566 10583 raft_consensus.cc:606] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:51.270694 10508 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:51.276158 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:51.276934 10583 raft_consensus.cc:3055] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:51.277750 10583 raft_consensus.cc:740] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:51.278906 10583 consensus_queue.cc:260] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.281450 10583 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.278160 11212 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.269408 10437 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.272217 17595 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:12:51.308935 10437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:51.311980 10437 raft_consensus.cc:606] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:51.313850  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:12:51.314570 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46332.7us
I20260709 12:12:51.316314  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:51.317257  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:51.320848 17588 raft_consensus.cc:1064] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:51.323225 17606 raft_consensus.cc:993] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:51.327527 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:51.328349 10438 raft_consensus.cc:3060] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.328508 17605 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.350471 17606 raft_consensus.cc:1081] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
W20260709 12:12:51.357509 10384 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:12:51.358023 10384 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 955c0a21e1e04f51bbbc7c06b5d611f5 already in progress
W20260709 12:12:51.358307 10384 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 955c0a21e1e04f51bbbc7c06b5d611f5 already in progress
I20260709 12:12:51.359776 10583 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:12:51.371083 17588 raft_consensus.cc:1068] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Service unavailable: leader transfer in progress
W20260709 12:12:51.382786 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46778.5us
I20260709 12:12:51.385159 10333 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.388459 11212 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.392760 17646 raft_consensus.cc:993] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:51.411132 17646 raft_consensus.cc:1081] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:51.394531 10434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48990
I20260709 12:12:51.414192 10434 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:51.414636 10434 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:51.401348 10478 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:51.423691 17606 raft_consensus.cc:2804] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:51.424393 17606 raft_consensus.cc:697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:51.425475 17606 consensus_queue.cc:237] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.429329 10332 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 9 to 10, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.442444 10438 raft_consensus.cc:2468] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:51.455327 10506 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:51.456338 10506 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:51.456854 10506 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:51.460872 10583 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.464411 10434 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:51.470005 17646 raft_consensus.cc:993] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:51.470705 17646 raft_consensus.cc:1081] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:51.501152 10434 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:51.514890 10508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:12:51.515694 10508 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:51.516258 10508 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:51.529326 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55398.6us
I20260709 12:12:51.540946 10506 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.550127 10438 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.557734 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:51.558775 10505 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:51.558508 10508 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.559361 10505 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:51.560472 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:51.561208 10587 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:51.567129 17642 raft_consensus.cc:993] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:51.567759 17642 raft_consensus.cc:1081] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:12:51.568717 10437 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:51.579866 10506 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:51.622804 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:51.623749 10583 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:51.624969 10583 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:51.624987 10587 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:51.571527 10505 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:51.627668 10508 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:51.629642 10505 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:51.633778 10583 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.637410 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:51.640193 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:51.640791 10438 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:51.641496 10583 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:51.643328 10508 raft_consensus.cc:1240] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:51.654028 10438 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:51.654332 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:51.655066 10434 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.662619 10583 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:12:51.666746 10587 raft_consensus.cc:3055] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:51.667414 10587 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:51.669973 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:51.676956 17595 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:51.677592 17595 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:51.687779 10434 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:51.689962 10479 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:51.666807 17534 consensus_queue.cc:1059] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 12:12:51.699206 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 153206us
I20260709 12:12:51.706017 10505 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:51.698002 17577 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:51.678596 17595 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:51.737852 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 63532.6us
I20260709 12:12:51.738757 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:51.731171 17577 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:51.760581 10434 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48986
I20260709 12:12:51.761183 10434 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:51.761487 10434 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:51.758720 10587 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.763739 10587 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.770771 10434 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:51.749244 17607 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:51.772142 10587 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:12:51.782287 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:51.782913 11212 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:51.790382 11212 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:51.791779 10404 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:51.793435 17527 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:51.793946 17527 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:51.794816 17527 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:51.802011 10434 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:51.826629 17648 raft_consensus.cc:993] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:51.829007 17648 raft_consensus.cc:1081] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:51.812391 17577 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.856585 10581 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:12:51.857311 10581 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:51.857725 10581 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.870915 10581 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.882275 10581 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:51.886415 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:51.887463 10511 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.893783 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:51.894913 10587 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:51.895448 10587 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:51.897156 10587 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:51.916062 10511 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:51.919842 10587 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:51.928223 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:51.933857 17466 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:51.934600 17466 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:51.935632 17466 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:51.901926 17607 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:51.933653 10587 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:51.980676 17607 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:52.075567 10587 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:52.078490 17578 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:12:52.103999 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 211658us
I20260709 12:12:52.106789 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:52.106976 10438 raft_consensus.cc:1240] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:52.107676 10437 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:52.108157 10437 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:52.109162 10437 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:52.121816 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 203100us
I20260709 12:12:52.129240 10431 raft_consensus.cc:1275] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:52.168396 17578 consensus_queue.cc:1048] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:12:52.310626 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 134759us
W20260709 12:12:52.357532 17671 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:12:52.359508 17670 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:12:52.464601 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 118341us
W20260709 12:12:52.595657 17677 log.cc:927] Time spent T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:12:52.601637 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 170876us
W20260709 12:12:52.603509 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 137506us
I20260709 12:12:52.629151 10579 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:52.635653 10433 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:52.612655  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:52.635888 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 284) took 1219 ms. Trace:
I20260709 12:12:52.665452 10579 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:12:52.668812 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 300) took 1263 ms. Trace:
I20260709 12:12:52.661445 10336 rpcz_store.cc:276] 0709 12:12:51.416540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.416810 (+   270us) service_pool.cc:224] Handling call
0709 12:12:52.635624 (+1218814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":285396,"reported_tablets":1}
I20260709 12:12:52.679550 10437 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:52.670293 10335 rpcz_store.cc:276] 0709 12:12:51.404808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.423797 (+ 18989us) service_pool.cc:224] Handling call
0709 12:12:52.668550 (+1244753us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":177707,"reported_tablets":2}
I20260709 12:12:52.699649 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 162) took 1269 ms. Trace:
I20260709 12:12:52.703963 10332 rpcz_store.cc:276] 0709 12:12:51.361376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.367135 (+  5759us) service_pool.cc:224] Handling call
0709 12:12:52.630756 (+1263621us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:51.581516 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:51.582171 (+   655us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:51.582205 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:12:51.582226 (+    21us) tablet.cc:662] Decoding operations
0709 12:12:51.582668 (+   442us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:51.582737 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:51.582767 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:51.607099 (+ 24332us) tablet.cc:701] Row locks acquired
0709 12:12:51.607126 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:12:51.647010 (+ 39884us) write_op.cc:270] Start()
0709 12:12:51.663825 (+ 16815us) write_op.cc:276] Timestamp: P: 1783599171640900 usec, L: 0
0709 12:12:51.663859 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:12:51.728777 (+ 64918us) log.cc:844] Serialized 591 byte log entry
0709 12:12:51.879273 (+150496us) op_driver.cc:464] REPLICATION: finished
0709 12:12:51.952255 (+ 72982us) write_op.cc:301] APPLY: starting
0709 12:12:51.960399 (+  8144us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:51.961683 (+  1284us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:51.961855 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:52.057155 (+ 95300us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:52.057353 (+   198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:12:52.057412 (+    59us) write_op.cc:312] APPLY: finished
0709 12:12:52.057710 (+   298us) log.cc:844] Serialized 25 byte log entry
0709 12:12:52.057932 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:52.058043 (+   111us) write_op.cc:454] Released schema lock
0709 12:12:52.058090 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:52.058644 (+   554us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:52.058716 (+    72us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":320872,"reported_tablets":4,"spinlock_wait_cycles":63744,"child_traces":[["op",{"apply.queue_time_us":69583,"apply.run_cpu_time_us":4506,"apply.run_wall_time_us":106334,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":140655,"prepare.run_cpu_time_us":3875,"prepare.run_wall_time_us":157210,"replication_time_us":212215,"thread_start_us":67141,"threads_started":1}]]}
I20260709 12:12:52.726635 10328 catalog_manager.cc:5697] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.745431 10579 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:52.756644 10328 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:12:52.770599 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.774267 10328 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 12 to 13, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.693039 10581 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:52.795682 17593 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:12:52.819197 17644 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:52.830976 11213 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:12:52.839890 10433 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:52.840067 10505 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:52.861900 10437 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:12:52.864655 10436 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:52.853973 10328 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 19, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.895109 10505 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:52.908475 17527 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:12:52.917552 11212 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:52.919946 17641 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:52.926090 17646 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:52.936995 17643 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:52.942961 11213 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Advancing to term 20
I20260709 12:12:52.989203 10586 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:52.996099 10507 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:53.009083 11213 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:53.017539 10507 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:53.044463 17588 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:53.058465 17686 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:12:53.113708 17658 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:12:53.100265 17680 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 12:12:53.178663 17667 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:53.188625 17686 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.189522 17686 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:53.193387 17686 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:53.206898 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:53.219481 10505 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
W20260709 12:12:53.194254 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 55973.2us
I20260709 12:12:53.228653 17607 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:53.243537 17686 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:53.247980 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:53.249586 17639 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:53.250088 17639 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.250456 17639 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:53.328966 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:53.348192 17674 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:12:53.399699 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 78242.2us
I20260709 12:12:53.420379 17648 raft_consensus.cc:1064] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:12:53.420992 17648 raft_consensus.cc:1068] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:12:53.422895 17466 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:12:53.440907 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104561us
W20260709 12:12:53.566920 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 288096us
I20260709 12:12:53.599833 11215 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:53.640738 17639 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:53.642975 10583 raft_consensus.cc:1240] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:12:53.649376 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62941.4us
W20260709 12:12:53.651556 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 244167us
I20260709 12:12:53.671375 17658 consensus_queue.cc:1059] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:53.699296 17527 raft_consensus.cc:1064] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:12:53.699431 17638 raft_consensus.cc:3055] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:53.702675 17638 raft_consensus.cc:740] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:53.703805 17638 consensus_queue.cc:260] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:53.705920 17638 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:53.733558 17639 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:53.746001 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1ecefe266eea48c4af129b425f391fe9 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:53.750537 10507 raft_consensus.cc:2955] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:53.779484 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 163) took 1034 ms. Trace:
I20260709 12:12:53.780251 10335 rpcz_store.cc:276] 0709 12:12:52.744754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.753952 (+  9198us) service_pool.cc:224] Handling call
0709 12:12:53.779373 (+1025421us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7,"reported_tablets":2}
I20260709 12:12:53.786327 10328 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 291) took 1254 ms. Trace:
I20260709 12:12:53.793007 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:53.802320 10505 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:53.796151 10332 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 20 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.833329 10332 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 15, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 15 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.819917 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:53.812491 10513 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:53.791159 10328 rpcz_store.cc:276] 0709 12:12:52.532397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.539532 (+  7135us) service_pool.cc:224] Handling call
0709 12:12:53.786240 (+1246708us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:53.015069 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:53.015894 (+   825us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:53.015951 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:12:53.015974 (+    23us) tablet.cc:662] Decoding operations
0709 12:12:53.016756 (+   782us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:53.016842 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:53.016877 (+    35us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:12:53.017471 (+   594us) tablet.cc:701] Row locks acquired
0709 12:12:53.017526 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:12:53.017762 (+   236us) write_op.cc:270] Start()
0709 12:12:53.017926 (+   164us) write_op.cc:276] Timestamp: P: 1783599173017684 usec, L: 0
0709 12:12:53.017950 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:12:53.068943 (+ 50993us) log.cc:844] Serialized 1738 byte log entry
0709 12:12:53.471359 (+402416us) op_driver.cc:464] REPLICATION: finished
0709 12:12:53.566136 (+ 94777us) write_op.cc:301] APPLY: starting
0709 12:12:53.566262 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:53.684265 (+118003us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:53.684383 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:53.709603 (+ 25220us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adc32a 00007f2c92a61f14 00007f2c92a61ec3 00007f2c92a61e38 00007f2c92a61df3 00007f2c92a610dc 00007f2c92a8ae16 00007f2c92a72cbb 00007f2c92a5d39d 00007f2c92a60d75
0709 12:12:53.743314 (+ 33711us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:53.743719 (+   405us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:12:53.743775 (+    56us) write_op.cc:312] APPLY: finished
0709 12:12:53.744205 (+   430us) log.cc:844] Serialized 49 byte log entry
0709 12:12:53.744457 (+   252us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:53.744618 (+   161us) write_op.cc:454] Released schema lock
0709 12:12:53.744699 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:53.745388 (+   689us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:53.745426 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46454,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":94458,"apply.run_cpu_time_us":9769,"apply.run_wall_time_us":179233,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"mutex_wait_us":13210,"num_ops":4,"prepare.queue_time_us":61327,"prepare.run_cpu_time_us":21009,"prepare.run_wall_time_us":145943,"raft.queue_time_us":165390,"raft.run_cpu_time_us":12600,"raft.run_wall_time_us":272947,"replication_time_us":453176,"spinlock_wait_cycles":3811584,"thread_start_us":149435,"threads_started":4,"wal-append.queue_time_us":55921}]]}
I20260709 12:12:53.854439 10434 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:12:53.873987 11215 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:12:53.877023 17606 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:12:53.890029 10583 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:12:53.896471 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:53.902963 17726 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:53.904657 17682 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:12:53.913918 17527 raft_consensus.cc:1068] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Unable to promote non-voter 52bae2d2caf94d97b1049ecd4f82a2df: Illegal state: Replica 99d42e7288f145f1bd5af212fd062995 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:53.995625 11212 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:12:54.004395 17606 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:54.011354 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 285) took 1005 ms. Trace:
I20260709 12:12:54.014197 10335 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.020659 17718 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Committing config change with OpId 21.16: config changed from index 15 to 16, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.011905 10332 rpcz_store.cc:276] 0709 12:12:53.005976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.006219 (+   243us) service_pool.cc:224] Handling call
0709 12:12:54.011282 (+1005063us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:53.955012 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:53.955773 (+   761us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:53.955817 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:12:53.955838 (+    21us) tablet.cc:662] Decoding operations
0709 12:12:53.956338 (+   500us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:53.956433 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:53.956470 (+    37us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:12:53.956903 (+   433us) tablet.cc:701] Row locks acquired
0709 12:12:53.956928 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:12:53.957103 (+   175us) write_op.cc:270] Start()
0709 12:12:53.957249 (+   146us) write_op.cc:276] Timestamp: P: 1783599173957049 usec, L: 0
0709 12:12:53.957274 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:12:53.958100 (+   826us) log.cc:844] Serialized 968 byte log entry
0709 12:12:53.962601 (+  4501us) op_driver.cc:464] REPLICATION: finished
0709 12:12:53.963136 (+   535us) write_op.cc:301] APPLY: starting
0709 12:12:53.963228 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:53.964992 (+  1764us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:53.965112 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:53.986883 (+ 21771us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:53.995551 (+  8668us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:12:53.995609 (+    58us) write_op.cc:312] APPLY: finished
0709 12:12:53.995868 (+   259us) log.cc:844] Serialized 33 byte log entry
0709 12:12:53.996074 (+   206us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:53.996185 (+   111us) write_op.cc:454] Released schema lock
0709 12:12:53.996227 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:53.996654 (+   427us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:53.996681 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":273,"apply.run_cpu_time_us":5120,"apply.run_wall_time_us":33465,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":113335,"prepare.run_cpu_time_us":3535,"prepare.run_wall_time_us":4290,"replication_time_us":5170,"thread_start_us":60593,"threads_started":1}]]}
I20260709 12:12:54.023154 10583 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 15 to 16, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.029986 10434 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:12:54.032384 17550 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:54.071766 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ecefe266eea48c4af129b425f391fe9 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:51568
I20260709 12:12:54.072858 10336 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:54.077632 11218 raft_consensus.cc:2955] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Committing config change with OpId 21.16: config changed from index 15 to 16, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.084422 10431 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:54.085871 10431 raft_consensus.cc:606] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:54.092708 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9042bec741aa476587972e688997da43"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:54.093425 10507 raft_consensus.cc:606] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:54.107951 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc98cd5edc9240fba53978ed59191cbd with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:54.121532 10507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:54.122350 10507 raft_consensus.cc:606] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:54.123805  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:12:54.126432  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:54.142692  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:54.157603 17643 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.173167 10434 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.186403 17593 raft_consensus.cc:993] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:54.216172 17593 raft_consensus.cc:1081] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:54.218027 17682 raft_consensus.cc:993] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:54.218570 17682 raft_consensus.cc:1081] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:54.220731 10585 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9042bec741aa476587972e688997da43"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:54.221468 10585 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:54.221889 10585 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:54.220561 10583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:54.228610 10583 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:54.229031 10583 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:54.265448 17734 tablet_replica.cc:333] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:12:54.254820 10335 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 15 to 16, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.285229 10585 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:54.316192 10585 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:54.323160 17680 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:54.324035 17680 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:54.327283 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:54.328226 11218 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:54.357152 10583 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:54.376242 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:54.379781 10583 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:54.387940 17680 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:54.394143 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:54.400103 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:54.400905 10509 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:54.401226 10509 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:54.402124 10509 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:54.403379 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:54.404024 10506 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:54.404340 10506 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:54.405156 10506 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:54.406529 10506 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:54.407996 17734 raft_consensus.cc:2243] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:54.408932 17734 raft_consensus.cc:2272] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:54.412637 11218 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:54.414671 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c61f4bfa2db8497dbf7e0446e2ab707d with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:12:54.418196 10509 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:54.442483 17734 ts_tablet_manager.cc:1916] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:54.450392 10403 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:54.452359 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet dc98cd5edc9240fba53978ed59191cbd with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:51568
I20260709 12:12:54.448814 11212 raft_consensus.cc:2955] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.473695 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:54.474550 10434 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:54.479194 10403 raft_consensus.cc:2686] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:12:54.489418 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:54.502678 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:54.503890 17686 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:54.504590 17686 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:54.505084 10335 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 10 to 11, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:12:54.505741 17686 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:54.506520 10506 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:54.494372 10509 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:54.517465 10552 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:54.518436 17715 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:54.519321 17715 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:54.520365 17715 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:54.524703 10434 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
W20260709 12:12:54.530094 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44683.5us
I20260709 12:12:54.554944 17680 raft_consensus.cc:993] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:54.556318 17680 raft_consensus.cc:1081] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:54.559703 10509 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:54.565127 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet c61f4bfa2db8497dbf7e0446e2ab707d with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:55454
I20260709 12:12:54.566562 10328 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 12 to 13, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 13 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.577812 10509 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:54.588620 10509 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:54.618340 17734 ts_tablet_manager.cc:1929] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.13
I20260709 12:12:54.618963 17734 log.cc:1199] T 1ecefe266eea48c4af129b425f391fe9 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/1ecefe266eea48c4af129b425f391fe9
I20260709 12:12:54.621413 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet 1ecefe266eea48c4af129b425f391fe9 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:54.621793 17734 tablet_replica.cc:333] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:12:54.618062 10328 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 14 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:54.638717 10509 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:54.659612 17734 raft_consensus.cc:2243] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:54.668820 17734 raft_consensus.cc:2272] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:54.669864 10509 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:54.671140 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:54.671900 10437 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:54.672267 10437 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:54.673182 10437 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:54.674193 10437 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:54.675341 17527 consensus_queue.cc:1175] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 17 status { last_received { term: 17 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 17 index: 12 } }
I20260709 12:12:54.676461 17738 tablet_replica.cc:333] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:12:54.679039 17738 raft_consensus.cc:2243] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:54.679986 17738 raft_consensus.cc:2272] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:54.683691 17738 ts_tablet_manager.cc:1916] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:54.672139 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:54.685377 10585 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:54.696496 10437 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:54.697840 10479 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:54.698840 17564 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:54.699519 17564 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:54.700438 17564 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:54.698814 17734 ts_tablet_manager.cc:1916] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:12:54.714305 10585 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:54.765872 17738 ts_tablet_manager.cc:1929] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 12:12:54.783901 10328 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 18, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.768571 17738 log.cc:1199] T c61f4bfa2db8497dbf7e0446e2ab707d P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/c61f4bfa2db8497dbf7e0446e2ab707d
I20260709 12:12:54.801899 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet c61f4bfa2db8497dbf7e0446e2ab707d (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:54.862466 17734 ts_tablet_manager.cc:1929] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.15
I20260709 12:12:54.863242 17734 log.cc:1199] T dc98cd5edc9240fba53978ed59191cbd P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/dc98cd5edc9240fba53978ed59191cbd
I20260709 12:12:54.867118 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet dc98cd5edc9240fba53978ed59191cbd (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:12:54.954677 10506 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:54.956410 17686 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:55.086203 10430 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:55.090147 10430 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:55.091832 17643 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:55.107714 11212 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:12:55.109390 10513 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:12:55.125084 17742 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:55.131736 17742 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:12:55.139938 17686 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:55.183327 17643 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:55.184084 17643 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:55.260124 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:55.261250 10431 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:12:55.273452 17643 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:55.287142 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53790.3us
I20260709 12:12:55.291497 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:55.293064 17564 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:55.293516 17564 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:55.293948 17564 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:55.345880 17564 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:56.930276 17761 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:56.931318 17761 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:56.937956 17762 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:56.938378 17761 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:56.939029 17762 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:56.942885 17762 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:56.945197 17763 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:56.946084 17762 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:56.946439 17763 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:56.946735 17762 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:56.950105 17762 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:56.944859 17761 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:56.951171 17761 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:56.957515 17763 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:56.963486 17768 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:56.972872 17761 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:56.973204 17760 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:56.973784 17760 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:56.989015 17771 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:56.986791 17767 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:56.989748 17771 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:56.992357 17771 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:56.989894 17767 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:56.997593 17760 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:56.997949 17767 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.002136 17761 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.002871 17761 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.005210 17773 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:12:57.010756 11212 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.653s	user 0.020s	sys 0.005s
I20260709 12:12:57.005517 17761 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.013095 17760 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:56.998956 17771 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.013705 17760 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.014389 17771 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.016136 17760 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.006510 17773 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.017378 17771 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.018347 17773 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.019315 17765 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.019896 17765 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:56.979946 17768 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.021776 17762 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.022186 17765 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.022326 17762 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.023877 17760 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.022754 17765 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.024505 17760 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.024690 17765 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.026932 17760 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.024777 17762 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.028998 17766 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:57.047719 11212 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.690s	user 0.022s	sys 0.005s
I20260709 12:12:57.035611 17770 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.060882 17765 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.062776 17762 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.062618 17766 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.063387 17779 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.064260 17760 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.063995 17770 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.060922 17777 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.069900 17775 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.070390 17773 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.075875 17775 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.076653 17773 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.077716 17771 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.078514 17775 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.078959 17773 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.022384 17768 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.078297 17771 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.080857 17770 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.073809 17762 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.083762 17762 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.081944 17771 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.075307 17760 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.085788 17773 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.086520 17773 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.087446 17760 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.089257 17773 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.089082 17777 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.090011 17773 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.091729 17777 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.090684 17773 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.092420 17779 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.095654 17771 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.096526 17771 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.097991 17766 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.100252 17773 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.101330 17771 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.094825 17779 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.106837 17745 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.107543 17745 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.109901 17766 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.110019 17745 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.110772 11212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57892 (request call id 1076) took 1753 ms. Trace:
I20260709 12:12:57.110487 17766 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:57.111189 11212 rpcz_store.cc:276] 0709 12:12:55.357100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.357247 (+   147us) service_pool.cc:224] Handling call
0709 12:12:55.357512 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.357716 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:57.049471 (+1691755us) log.cc:844] Serialized 14 byte log entry
0709 12:12:57.109261 (+ 59790us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:57.109319 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:57.109392 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.110529 (+  1137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.110679 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":517666,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":83,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":140,"thread_start_us":1750004,"threads_started":2,"wal-append.queue_time_us":57627}
I20260709 12:12:57.112789 17766 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.114389 17781 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.113749 17778 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.110773 17764 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.115022 17781 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.115290 17778 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.115697 17764 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.114742 17766 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.117532 17781 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.117542 17778 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.117928 17766 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.122678 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.123090 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.123411 10510 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 11.
I20260709 12:12:57.123576 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.124226 10508 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:12:57.124904 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.126055 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.126400 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.126762 10511 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:12:57.126924 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.127125 10508 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:57.127552 10510 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:57.127686 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.135149 10511 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:57.128743 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.138885 17778 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.139575 17778 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.140059 17777 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.140693 17777 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.141980 17778 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.129483 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.126318 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.144263 10512 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:57.144419 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.145100 17768 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:57.145540 17768 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.146009 17768 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:57.131748 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.146541 17782 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.146894 10506 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:57.147249 17782 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.149083 17745 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.149735 17745 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.128017 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.152333 10508 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:12:57.143690 17777 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.153285 17774 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:57.151997 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.154349 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.174896 17781 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.120757 17766 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.176899 17765 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:57.177474 17765 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.177989 17765 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:57.178395 17781 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.181366 17781 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.176307 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.149740 17782 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.175616 17774 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.183696 17777 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.177999 17787 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.197685 17779 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:57.201864 17779 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.202270 17779 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:57.203382 17783 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.203974 17783 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:57.204736 10430 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.859s	user 0.095s	sys 0.165s
W20260709 12:12:57.206862 10430 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.861s	user 0.095s	sys 0.165s
I20260709 12:12:57.206768 17783 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.195040 17770 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.196305 17762 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.216233 17785 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.209285 17564 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.199846 17774 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:57.217509 17777 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.211377 17784 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.222908 17745 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.219069 17787 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.287786 10430 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 4247) took 1943 ms. Trace:
I20260709 12:12:57.222112 17781 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.217422 17778 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.224460 17782 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.299525 17768 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.235631 17550 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.303805 10430 rpcz_store.cc:276] 0709 12:12:55.344627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.345115 (+   488us) service_pool.cc:224] Handling call
0709 12:12:55.345405 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.345619 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:57.217858 (+1872239us) log.cc:844] Serialized 14 byte log entry
0709 12:12:57.286707 (+ 68849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:57.286818 (+   111us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:57.286918 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.287406 (+   488us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.287635 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17097,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":148,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":156,"spinlock_wait_cycles":81664,"thread_start_us":1939647,"threads_started":2,"wal-append.queue_time_us":68201}
I20260709 12:12:57.207490 17765 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.319500 17766 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.320297 17766 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.324532 17766 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.291134 17770 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.340651 17564 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.342319 17796 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.309274 17745 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.344977 17797 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.345198 17794 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.350509 17797 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.350620 17794 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.351480 17793 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.352082 17793 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.317744 17778 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.333545 17550 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.292397 17785 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.355573 17550 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.349617 17745 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.302754 17762 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.338306 17766 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.367406 17798 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.367879 17760 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.368233 17798 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.368801 17760 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.316843 17781 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.302951 17784 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.318833 17782 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.349527 17796 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.383234 17786 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.384084 17796 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.384829 17766 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.384378 17786 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.391192 17788 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.391872 17788 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:57.349267 17564 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:57.393231 10478 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.139s	user 0.002s	sys 0.035s
W20260709 12:12:57.455071 10478 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.201s	user 0.005s	sys 0.035s
I20260709 12:12:57.397629  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:57.398430 17797 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.464025 17564 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.412196 17787 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.444528 17784 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.485098 17784 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.485869 17784 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.394408 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.488410 17764 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.492687 17788 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.507436 17799 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.508314 17799 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.507440 17790 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.509505 17790 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.510893 17799 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:57.515125 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 120807us
I20260709 12:12:57.516063 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.519183 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:12:57.573212 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 54097.4us
I20260709 12:12:57.511794 17790 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.434610 17779 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.581024 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.582830 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.578671 17799 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:57.575747 17797 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:57.579272 17787 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.582545 17790 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:57.583863 17797 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.589205 17550 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.591248 17789 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.592355 17550 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.591495 17787 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.595009 17797 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.596612 17787 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.598428 17779 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.592468 17789 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.584336 17799 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.589936 17790 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.594803 17550 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.600747 17790 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.599197 17779 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:57.609251 17779 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.615938 17779 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.626072 17779 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.638317 17779 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.643718 17787 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.644472 17779 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:57.644973 17779 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.645385 17779 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:57.627372 17784 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.603443 17796 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.601598 17799 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.635289 17800 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.640478 17782 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.641465 17777 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.641789 17564 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.632249 17794 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.670011 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.670904 10580 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:57.647524 17764 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.672099 17764 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.674571 17764 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.675225 17793 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.664937 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:57.678679 17760 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.668468 17796 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.681322 17796 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.668169 10585 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:57.673012 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.716339 10587 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
I20260709 12:12:57.692232 17794 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.723675 17794 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.726114 17794 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.644589 17787 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.722812 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:57.727463 10587 raft_consensus.cc:2393] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:57.728165 17766 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.728303 17787 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.729295 17799 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.660449 17800 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.731422 10430 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
W20260709 12:12:57.731968 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 142490us
I20260709 12:12:57.661044 17774 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.765700 17789 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.692483 17564 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.773172 17564 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.739220 17796 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.774312 17796 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.776657 17796 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.777626 17790 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.779911 17790 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.781184 17798 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.782590 17770 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.784858 17774 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.787392 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.788067 10585 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 22.
I20260709 12:12:57.788972 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:57.790174 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50913.2us
I20260709 12:12:57.798933 17785 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.802184 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.804107 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.805083 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.807331 17786 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.807431 17772 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.808626 17768 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.809870 17781 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.811094 17762 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.811251 17800 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.798718 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.814268 17788 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.835357 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.836054 17564 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.835817 17788 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.834405 17778 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.836627 17564 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.837146 17796 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.837682 17796 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.838308 17788 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.839598 17564 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.839133 17772 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.840305 17796 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.841483 10431 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:57.843374 10431 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:57.843834 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.846781 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.846949 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.847730 17799 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.847694 10586 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:57.812419 17765 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.834327 17783 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.849105 17783 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.829058 17745 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.814558 17789 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.770644 17797 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.888088 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.872258 17784 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:57.901153 17783 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.901546 17784 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.889477 17808 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.903203 17808 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.903417 17789 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.904024 17784 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.888552 10438 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:57.906140 17808 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.887547 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.907166 17810 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.907387 10506 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:12:57.907783 17810 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.908422 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.908922 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.909598 10505 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:12:57.909780 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.910276 17810 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.910837 17810 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.911098 17764 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:57.850612 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.911465 17810 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.911760 17764 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:57.912248 10583 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:12:57.912521 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.906037 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.914476 10504 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:57.872391 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.916319 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.851933 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.910624 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.919060 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.919060 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.914182 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.920213 10511 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:12:57.918175 10431 raft_consensus.cc:2410] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 12:12:57.921502 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45561.9us
I20260709 12:12:57.921527 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.925407 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.926136 10434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:12:57.928433 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.928833 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.923229 17765 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.929993 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.929895 17765 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.930769 10435 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:57.932331 17765 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.924204 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.926927 10434 raft_consensus.cc:606] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:57.933530 10479 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:57.938942 17799 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:57.943917 10433 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:57.930037 10580 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:57.944396 17799 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:57.902745 17797 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.975500 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.974925 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.981290 10585 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:57.974617 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.948067 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.947137 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:57.960597 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.949532 17764 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:57.992690 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:57.993700 10509 raft_consensus.cc:606] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:12:57.994326 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:57.995689 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.996888 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.997323 17783 raft_consensus.cc:2804] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:57.997932 17783 raft_consensus.cc:697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:57.956615 17789 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:57.923579 17784 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:57.999361 17784 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.001695 17784 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:57.999078 17783 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:12:57.995653 10429 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:12:57.994405 17810 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.003665 10434 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
W20260709 12:12:57.996853 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63418.2us
I20260709 12:12:58.008034 10434 raft_consensus.cc:606] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:57.910214 17808 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:58.010200 11245 debug-util.cc:398] Leaking SignalData structure 0x7b080029c880 after lost signal to thread 9264
I20260709 12:12:58.010380 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.993114 17799 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.011327 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.854933 17772 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:58.166612 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.189582 10436 raft_consensus.cc:2410] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:12:58.172792 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.189631 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:57.997046 17779 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.004398 17765 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.175915 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:58.057767 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.174185 17785 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:58.198944 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:58.171511 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 160362us
I20260709 12:12:58.201226 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.172103 17808 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:58.181144 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.202912 17789 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:58.179373 17810 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:58.173372 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.195041 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.170840 17806 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:58.181979 11245 debug-util.cc:398] Leaking SignalData structure 0x7b080027b220 after lost signal to thread 10370
I20260709 12:12:58.188700 17793 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:58.214483 11245 debug-util.cc:398] Leaking SignalData structure 0x7b0800323fe0 after lost signal to thread 10462
I20260709 12:12:58.208338 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.213071 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.209390 10433 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:58.215678 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.216926 10434 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:58.217756 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.218447 11212 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
W20260709 12:12:58.219000 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1180 after lost signal to thread 10537
W20260709 12:12:58.220242 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08003bdaa0 after lost signal to thread 10611
I20260709 12:12:58.214316 17810 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:58.229588 17779 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.200892 17765 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:58.232419 17765 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:58.210240 17785 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.236086 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:58.240600 11245 debug-util.cc:398] Leaking SignalData structure 0x7b08003ef500 after lost signal to thread 11248
I20260709 12:12:58.244493 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.226704 17810 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:58.248683 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.250921 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.254133 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.259683 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.261610 10434 raft_consensus.cc:1240] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:58.263103 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.263322 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.260098 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.260803 10552 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.267237 17745 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:58.267627 17745 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.268003 17745 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:58.270362 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.270714 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.247983 17785 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.272331 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.273128 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:58.275611 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.272243 10552 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.276014 11215 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:58.277091 17809 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.277302 10552 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.277550 17809 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.278018 17809 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.278262 17788 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.278847 17788 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:58.279363 17788 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.285671 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.287197 10433 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:58.287339 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.287982 10431 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 11.
I20260709 12:12:58.288671 10436 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:58.250064 17793 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.289713 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.290023 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.290378 10429 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:12:58.291335 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.291640 17793 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.293123 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.294456 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.295473 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.296715 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.297189 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.248162 17806 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.300740 10434 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 13.
I20260709 12:12:58.301964 17806 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.307934 17808 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:58.308612 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.308683 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.302858 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:58.309387 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 107997us
I20260709 12:12:58.310125 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.310664 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.311456 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.317764 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.320622 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.322189 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.323673 17807 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:12:58.324160 17807 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.324573 17807 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:58.326193 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.317076 17787 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:58.327365 17787 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.327775 17787 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:58.329200 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.329458 17745 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:58.332672 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.337829 10511 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:58.340464 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.341574 17564 raft_consensus.cc:993] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:58.342226 17564 raft_consensus.cc:1081] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:58.342389 17745 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:58.343421 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.343710 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.352639 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.359548 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.347749 10336 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 20 to 21, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.364595 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.370679 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.365500 17781 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:12:58.376024 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.373098 17745 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:58.365782 17786 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:12:58.384124 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.388566 17786 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:58.389164 17786 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:58.389847 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:58.374428 17782 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:58.402201 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.406000 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.409250 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.414884 17778 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:58.422811 17778 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:58.424705 17778 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:12:58.428573 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51226.2us
I20260709 12:12:58.429862 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:58.431874 17784 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:58.432260 17784 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.432660 17784 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:58.432345 17788 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.435083 10404 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:58.435730 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.430709 17772 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:58.461493 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.461787 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.462272 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.463372 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.464138 17772 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:58.464540 17772 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:58.465324 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.465526 17772 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.436400 17785 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:12:58.437376 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 121050us
I20260709 12:12:58.467603 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.468478 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.469058 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.469632 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.470796 10404 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:58.471709 17765 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:58.467407 17785 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:58.472450 17785 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:58.474083 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.475728 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.415796 10586 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:58.444406 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.445556 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.448031 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:58.477442 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 146120us
I20260709 12:12:58.449440 17809 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:58.451705 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.481390 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.481946 11214 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:12:58.455801 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.482524 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.456137 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.456569 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.456790 17788 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:58.457437 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.485078 10585 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:58.458854 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.459200 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.459561 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.459849 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.460067 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.460605 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.461097 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.463490 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.390339 17786 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:58.479422 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.479558 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:58.484498 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48859.1us
I20260709 12:12:58.494946 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.496217 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.494891 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.488404 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.500126 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.500851 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.483889 17772 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.502522 17789 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:58.503188 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.508205 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.513691 17787 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:58.515218 10504 raft_consensus.cc:1240] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:58.517151 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.518791 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.479162 17778 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:58.519681 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.525023 17788 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:58.527765 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.533806 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.534966 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.535822 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.536342 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.527909 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.532954 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.538590 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.540138 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.485205 17766 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:58.540438 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.528209 17787 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:58.541368 10437 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:58.542346 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.551606 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.543809 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.547132 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.553347 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.555455 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.501655 17809 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:58.558945 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.560496 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.558966 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.564706 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.568790 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.571703 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.574014 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.575054 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.558005 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:58.598070 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.599447 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.600134 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.600536 10504 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 17: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:12:58.600986 10553 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.602618 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.603791 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.604280 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.605710 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.605794 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.607862 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.609030 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.610620 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.610918 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:58.550892 17786 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:58.613576 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.617149 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.614176 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.615312 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.608906 17800 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:58.620956 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.621727 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.623046 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.623872 17816 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:58.626066 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:58.627271 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.567665 17745 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:58.626730 10587 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:58.637138 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:12:58.639065 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 115861us
I20260709 12:12:58.611953 17788 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:58.549815 17777 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:12:58.612655 17766 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:12:58.654963 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 97177.8us
I20260709 12:12:58.657078 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:12:58.659044 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 109723us
I20260709 12:12:58.659895 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.665688 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.668272 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:12:58.672554 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 62020.5us
I20260709 12:12:58.676415 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.655038 17788 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.674763 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:58.692539 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:58.707139 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:58.712136 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.713862 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:58.715904 17815 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:58.716187 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:58.716423 17815 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.716952 17815 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:58.728978 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 54333.1us
I20260709 12:12:58.635289 17784 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:58.734566 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 73794.2us
I20260709 12:12:58.797598 11178 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.797672 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:58.809728 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 141545us
I20260709 12:12:58.812142 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:58.725970 17788 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:58.725729 17800 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:58.828394 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:58.768955 17816 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:58.764097 17825 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:58.833024 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 176094us
W20260709 12:12:58.837633 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 202557us
I20260709 12:12:58.885632 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:58.887086 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59313.6us
I20260709 12:12:58.892685 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:58.798336 17808 consensus_queue.cc:1059] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:12:58.806614 17811 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
W20260709 12:12:58.885967 17807 consensus_meta.cc:287] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.002s
W20260709 12:12:58.896286 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 236464us
I20260709 12:12:58.826690 17766 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
W20260709 12:12:58.904112 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71694us
I20260709 12:12:58.904524 17828 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:58.835465 17821 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:58.834504 17816 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.912984 17745 consensus_queue.cc:1059] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:12:58.885196 17827 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:58.896108 17782 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:58.928959 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.937435 10478 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:12:58.833189 17809 consensus_queue.cc:1059] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:12:58.917368 10582 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:12:58.928417 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:58.938740 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.941399 17822 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.943431 17784 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:58.951515 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 152332us
I20260709 12:12:58.965066 11178 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.953832 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:58.931529 17828 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.935057 17816 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.003485 17835 raft_consensus.cc:993] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:59.004201 17835 raft_consensus.cc:1081] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:12:59.005898 17838 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:59.006398 17838 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:58.948539 17820 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:59.010529 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 116499us
I20260709 12:12:58.967501 17787 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:12:59.012930 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84588.4us
I20260709 12:12:58.952288 17822 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:58.930671 17807 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.022650 17796 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:59.023329 17796 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.024362 17824 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:59.026279 17796 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:58.997241 17828 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:59.020474 17822 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.010365 17820 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:59.034796 17820 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.019420 17811 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:59.041997 10508 raft_consensus.cc:1240] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 22. Current term is 23. Ops: []
I20260709 12:12:59.043571 17807 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:59.046149 17785 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.002s
I20260709 12:12:59.009887 17838 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.048250 17838 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.053719 10583 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
W20260709 12:12:59.055188 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 162569us
I20260709 12:12:59.057597 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:12:59.060254 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 43768.8us
I20260709 12:12:59.061841 11178 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.062438 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.065402 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.066457 17821 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.066635 17811 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.066478 17826 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.068176 17839 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:59.070226 17808 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:59.071399 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.071739 17829 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:59.072362 17829 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.073257 17829 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.073680 10579 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:12:59.071700 17819 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.076817 10579 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:59.077224 10579 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.077548 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.077924 17822 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.079164 17829 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:59.081907 17832 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:59.083693 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.085299 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.085844 10585 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:59.087136 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.082366 17832 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.088444 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.089035 17832 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.089426 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:59.090158 17832 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:59.091552 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.092191 17815 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.094355 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.099561 10580 raft_consensus.cc:2393] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:12:59.103106 17788 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.103896 17820 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.105846 17822 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.106184 17820 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:59.046849 17785 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.108683 10434 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:59.113443 17781 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:59.113865 17781 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.114617 17815 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.114842 17781 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.116057 17781 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:59.118209 17836 raft_consensus.cc:3055] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:59.118705 17836 raft_consensus.cc:740] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.119694 17819 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:59.119794 17836 consensus_queue.cc:260] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.120369 17819 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:12:59.121287 17836 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:59.122849 17819 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:12:59.123241 11218 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:59.123330 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.125070 10508 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:12:59.125317 17824 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:59.125710 10508 raft_consensus.cc:606] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.126432 17827 raft_consensus.cc:3055] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:59.126870 17827 raft_consensus.cc:740] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.127808 17827 consensus_queue.cc:260] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.129350 17827 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:59.129475 17785 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.125711 17824 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:59.130653 17824 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.131459  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:59.115268 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:12:59.133693  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:59.134588  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:59.136109 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 79825.8us
I20260709 12:12:59.137189 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.138947 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:59.159068 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43876.9us
I20260709 12:12:59.160825 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.162959 10434 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:59.164791 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.166527 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.179750 17782 raft_consensus.cc:3055] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:59.185221 17782 raft_consensus.cc:740] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:12:59.185544 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.179924 17779 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:59.171722 17785 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:59.193320 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.195472 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.180159 17839 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:59.204596 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:59.201678 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:12:59.200027 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 183654us
W20260709 12:12:59.205333 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66540.9us
I20260709 12:12:59.206010 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.201861 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.202741 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.207026 10583 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 18: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:12:59.187909 17778 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.002s
I20260709 12:12:59.208871 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.211261 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.153810 17800 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:12:59.216770 11217 raft_consensus.cc:3060] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Advancing to term 21
I20260709 12:12:59.219854 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:59.220304 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.220924 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:12:59.212781 11177 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.66:44681 timed out after 1.575s (SENT)
I20260709 12:12:59.204001 17825 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.207329 17785 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:59.237201 17843 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:59.239321 17792 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:12:59.240223 17851 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:12:59.240741 17842 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:12:59.211450 17835 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.215123 17788 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:12:59.243194 17845 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:12:59.243655 17847 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:59.231396 17839 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:59.209513 17745 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.247328 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.233736 17828 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.248672 17778 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:59.249351 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:59.276825 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.276925 10579 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.187927 17782 consensus_queue.cc:260] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.296078 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:12:59.297372 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:59.297216 17785 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.299829 10432 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:12:59.323920 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 74665.6us
I20260709 12:12:59.306151 11217 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:59.281985 17843 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:12:59.310154 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104197us
I20260709 12:12:59.317233 10587 raft_consensus.cc:1240] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:59.325215 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.328913 10579 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:59.320447 17793 raft_consensus.cc:993] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:59.329249 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
W20260709 12:12:59.327093 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 117824us
I20260709 12:12:59.319387 17778 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:12:59.333359 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.334848 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.344319 17850 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.345324 17778 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:59.347469 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:59.348218 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.349794 11213 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:12:59.350446 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.351914 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.348093 10513 raft_consensus.cc:3055] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:59.352612 10513 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:59.354362 10578 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 12.
I20260709 12:12:59.329248 17843 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.329496 17842 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:59.366214 17842 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:59.366816 17842 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.308079 17851 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.371246 17820 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.373545 17822 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:59.373996 17822 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:59.356065 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.357968 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.360072 17777 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:59.380925 10479 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.354528 17806 raft_consensus.cc:1081] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:12:59.382699 10479 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.312486 17847 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.379930 17777 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.387430 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.356770 17793 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.388758 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.358423 17792 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:59.383857 17807 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:59.392185 17833 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:12:59.394488 17804 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.396557 17816 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.397723 17828 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.397558 17824 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.400024 17824 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:59.400628 10431 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:59.400592 17745 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:59.403673 17792 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:59.404106 17792 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.405807 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.406392 10479 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.386615 10513 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.387336 17779 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.408245 10513 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.409240 10479 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.410341 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.407239 17784 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:59.416705 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.417341 10479 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.417644 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.412328 17819 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.419229 17804 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:12:59.418608 17825 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.411531 10504 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:12:59.424821 17847 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:59.428414 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:12:59.427577 17825 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.411446 17564 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:59.429819 17777 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.431217 17777 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:59.426297 17772 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:59.435364 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.435483 17851 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:59.436200 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.436228 17772 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.436913 10553 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.437696 17772 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.438198 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.438946 17772 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.441093 17787 raft_consensus.cc:993] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:12:59.441581 17787 raft_consensus.cc:1081] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
W20260709 12:12:59.447149 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 234392us
W20260709 12:12:59.447715 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.447965 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.452392 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:59.457324 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.458415 17827 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:12:59.459944 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.460839 17827 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.462165 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.463096 17827 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.398883 17816 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.469017 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.469903 17829 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:59.470252 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.471937 11177 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.472782 17827 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:59.474232 11177 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.474380 17822 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.475109 17827 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.475998 17822 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:59.476953 11177 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.477839 17827 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.445592 17847 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.479108 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.480173 17827 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.480432 17823 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.481989 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.482865 17827 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.485355 17827 consensus_queue.cc:1059] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:12:59.486686 17829 raft_consensus.cc:3055] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:12:59.474611 17778 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.487097 17829 raft_consensus.cc:740] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.487140 17816 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.488129 17829 consensus_queue.cc:260] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.489231 17829 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:59.489884 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 300) took 1206 ms. Trace:
I20260709 12:12:59.492802 17778 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:59.493415 10584 raft_consensus.cc:1240] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:59.493690 10580 raft_consensus.cc:1240] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:59.490327 10336 rpcz_store.cc:276] 0709 12:12:58.283783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.283962 (+   179us) service_pool.cc:224] Handling call
0709 12:12:59.489773 (+1205811us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:58.616359 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:58.622233 (+  5874us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:58.622270 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:12:58.622291 (+    21us) tablet.cc:662] Decoding operations
0709 12:12:58.622667 (+   376us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:58.622742 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:58.622774 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:12:58.622973 (+   199us) tablet.cc:701] Row locks acquired
0709 12:12:58.623034 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:12:58.623184 (+   150us) write_op.cc:270] Start()
0709 12:12:58.623315 (+   131us) write_op.cc:276] Timestamp: P: 1783599178623138 usec, L: 0
0709 12:12:58.623337 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:12:58.624162 (+   825us) log.cc:844] Serialized 591 byte log entry
0709 12:12:59.185853 (+561691us) op_driver.cc:464] REPLICATION: finished
0709 12:12:59.336606 (+150753us) write_op.cc:301] APPLY: starting
0709 12:12:59.336693 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:59.337775 (+  1082us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:59.337861 (+    86us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:59.338433 (+   572us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:59.338587 (+   154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:12:59.338617 (+    30us) write_op.cc:312] APPLY: finished
0709 12:12:59.338788 (+   171us) log.cc:844] Serialized 25 byte log entry
0709 12:12:59.338989 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:59.339106 (+   117us) write_op.cc:454] Released schema lock
0709 12:12:59.339145 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:59.339550 (+   405us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:59.339589 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":150508,"apply.run_cpu_time_us":2921,"apply.run_wall_time_us":2925,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":171851,"prepare.run_cpu_time_us":8905,"prepare.run_wall_time_us":258248,"raft.queue_time_us":263781,"raft.run_cpu_time_us":7272,"raft.run_wall_time_us":143155,"replication_time_us":562295,"thread_start_us":714159,"threads_started":4,"wal-append.queue_time_us":295486}]]}
I20260709 12:12:59.497365 10580 raft_consensus.cc:1240] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:59.502844 17830 consensus_queue.cc:1059] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:59.504479 17849 raft_consensus.cc:3055] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:59.504914 17849 raft_consensus.cc:740] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.505198 10578 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:59.505857 17849 consensus_queue.cc:260] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:12:59.506724 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:59.508180 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:59.515094 11175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.515295 17792 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.515113 17782 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:59.517729 17792 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.518399 17849 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:59.477355 10513 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
W20260709 12:12:59.531270 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.533087 10552 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:12:59.509893 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 92715.3us
I20260709 12:12:59.510622 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:59.536401 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.541309 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:59.522118 17851 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.543530 17816 raft_consensus.cc:2804] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:59.544217 11217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57904
W20260709 12:12:59.544550 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 72490.4us
I20260709 12:12:59.544934 11217 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:59.511693 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.545333 11217 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.545486 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.547490 10508 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:59.548333 17816 raft_consensus.cc:697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.548170 17827 consensus_queue.cc:1059] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:59.548820 17842 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.549583 17816 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.550277 17827 raft_consensus.cc:3055] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:59.550658 17827 raft_consensus.cc:740] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.548215 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.548401 17785 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.553782 17842 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.554517 17820 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:59.557683 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.560654 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.561287 17839 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.558547 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.562235 17832 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:59.563503 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51815.5us
I20260709 12:12:59.574728 10553 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.570181 10335 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 17 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:59.575093 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.579725 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.580885 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.584296 17836 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:59.559831 17843 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.585426 17848 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:12:59.586751 17848 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:59.587148 17848 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.593564 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.590874 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.573850 17785 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:59.593997 11217 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.600656 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.601930 10479 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.559188 17830 consensus_queue.cc:1059] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:12:59.608852 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:59.592114 17839 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.568524 17820 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.613008 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.613646 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.595654 17777 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.598738 17766 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.598160 17792 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:59.601848 17793 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:59.617959 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c545de01b7fb4c89946871c1d522006f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:59.625036 17807 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:59.624055 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.628949 10578 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.629896 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.620011 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.626407 11217 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:59.636656 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.637430 17844 raft_consensus.cc:3055] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:59.637879 17844 raft_consensus.cc:740] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.639175 17844 consensus_queue.cc:260] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.640570 17844 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:59.641368 17820 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:59.611955 17825 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:59.611528 17833 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:12:59.575637 17827 consensus_queue.cc:260] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.655239 17827 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:59.616420 17851 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.622457 17823 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.660159 17819 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:59.660619 17819 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:59.661553 17819 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.663363 17819 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.673910 17793 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:59.676685 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56728us
I20260709 12:12:59.588162 17848 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.622900 17564 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.679338 10504 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:59.680117 17843 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:12:59.647640 10578 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
I20260709 12:12:59.682477 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:59.688089 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64118.9us
I20260709 12:12:59.684746 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:59.689760 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.690109 10579 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:59.692719 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:12:59.693929 17804 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:59.694276 10587 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:59.694916 10579 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:12:59.695236 17847 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:59.695559 10579 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:59.696075 10579 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.696367 17782 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:59.696954 17782 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.695271 17842 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:59.698101 17782 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.701128 11178 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.702492 11178 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.703583 17782 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.696049 17849 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:59.700081 17848 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.691834 11177 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:59.714529 17829 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:59.715461 17829 raft_consensus.cc:2804] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:59.715993 17829 raft_consensus.cc:697] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:12:59.716995 17829 consensus_queue.cc:237] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.717262 17820 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:59.719260 17842 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:59.719677 17842 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:59.720711 17842 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.722884 11178 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.724699 17842 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.731031 17841 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:59.731556 17849 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:59.732991 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.734350 10580 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:59.734582 17844 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:59.738701 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:59.739385 17836 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:59.745718 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.748237 10504 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 15.
I20260709 12:12:59.750159 11215 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:12:59.753499 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.759097 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:59.759296 10508 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:12:59.758915 10579 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.759934 10508 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:59.760356 10508 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.761690 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:12:59.761835 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 69210us
I20260709 12:12:59.762610 10581 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:59.763027 11178 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.763180 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.764628 10553 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:12:59.761258 17851 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:12:59.767959 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.773769 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:59.771646 17816 raft_consensus.cc:2804] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:59.767436 17851 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.779274 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:59.774861 10504 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:12:59.780189 17851 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.770605 17804 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.784291 10431 raft_consensus.cc:2393] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 12: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:59.785777 10579 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:12:59.786938 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:12:59.766276 17827 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.789376 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.792502 17804 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:12:59.793035 17804 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.794098 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:12:59.794687 10504 raft_consensus.cc:3055] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:59.767819 17844 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.794976 10504 raft_consensus.cc:740] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:12:59.798084 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.804658 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:12:59.839047 10504 consensus_queue.cc:260] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:12:59.767634 17849 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.842332 10504 raft_consensus.cc:3060] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.767562 17792 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:59.843849 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:12:59.845705 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:12:59.851859 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47326.7us
I20260709 12:12:59.852756 10579 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 18: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:59.791934 17745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:50898 (request call id 4) took 1877 ms. Trace:
I20260709 12:12:59.854802 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.855872 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:12:59.785742 17816 raft_consensus.cc:697] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:12:59.857457 11217 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.857753 10504 raft_consensus.cc:2468] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
I20260709 12:12:59.857764 10479 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:12:59.860926 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:12:59.862078 11220 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:12:59.866544 10336 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 21 to 22, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:59.857884 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:12:59.874497 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 89314.7us
I20260709 12:12:59.875705 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:12:59.888615 10429 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:12:59.847558 17804 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:12:59.902479 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.857235 17820 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.865165 17787 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:59.872290 17844 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.782836 17830 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:12:59.894805 17785 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:12:59.920691 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60964.2us
I20260709 12:12:59.904740 17799 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:59.923281 10508 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.936368 10508 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:12:59.900718 17792 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:12:59.928653 17785 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:59.955148 11217 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
I20260709 12:12:59.933743 17799 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:12:59.966619 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:12:59.968430 10581 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.929847 17824 consensus_queue.cc:1059] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:12:59.986835 17785 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:12:59.858392 17745 rpcz_store.cc:276] 0709 12:12:57.908393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.913536 (+  5143us) service_pool.cc:224] Handling call
0709 12:12:57.998616 (+ 85080us) log.cc:844] Serialized 492 byte log entry
0709 12:12:58.267203 (+268587us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b540042f020. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:12:58.324431 (+ 57228us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b18002a6400. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a5fb26 00007f2c92a5949a 00007f2c92a58714
0709 12:12:58.325002 (+   571us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b540042f020. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a5fb26 00007f2c92a5949a 00007f2c92a58714 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634
0709 12:12:58.382399 (+ 57397us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b18002a6400. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:12:58.417065 (+ 34666us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b540042f020. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:12:59.784308 (+1367243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":674,"raft.run_cpu_time_us":10839,"raft.run_wall_time_us":280970,"spinlock_wait_cycles":78051328,"thread_start_us":237165,"threads_started":1,"wal-append.queue_time_us":266558,"wal-append.run_cpu_time_us":2490,"wal-append.run_wall_time_us":1024611}
I20260709 12:13:00.007424 17789 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:00.008239 17789 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.011726 17827 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:00.012440 17827 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:00.014173 17827 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.027976 17827 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:00.029429 17799 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.029480 17789 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:00.034619 17863 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:12:59.939330 17816 consensus_queue.cc:237] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.037606 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:00.038396 11218 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:00.062062 11177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.053522 10328 catalog_manager.cc:5697] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 21 to 22, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 22 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:00.067521 11214 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:00.074406 10504 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
W20260709 12:13:00.076618 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.066448 17745 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:00.118844 10553 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:13:00.127488 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 222940us
W20260709 12:13:00.135084 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.104732 17820 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:00.153088 10504 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:13:00.144845 17766 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:00.105904 17793 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.055s
I20260709 12:13:00.144770 17745 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:00.183525 10581 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:00.103994 17792 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.187532 17825 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:13:00.211100 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:00.168560 17820 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.191316 17766 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:00.216252 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:00.220500 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 74749.5us
I20260709 12:13:00.171941 17836 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:13:00.214797 10504 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:00.237658 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.237831 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.238826 10431 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:00.239473 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.239758 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.240363 10437 raft_consensus.cc:3060] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:00.241811 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.242148 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.242467 10431 raft_consensus.cc:3060] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:13:00.242663 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:00.242823 10430 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:00.244270 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:00.245618 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.246774 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.247220 10508 raft_consensus.cc:1240] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 17. Current term is 19. Ops: []
I20260709 12:13:00.247845 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.248467 10433 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:00.248518 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:00.248893 10433 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:00.249007 10508 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 15. Current term is 16. Ops: []
I20260709 12:13:00.249483 10513 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:00.250042 17830 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:00.249774 10433 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.251469 10433 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:00.257007 10436 raft_consensus.cc:1240] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:00.213953 17825 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:13:00.262813 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 101144us
I20260709 12:13:00.262925 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:00.324427 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:00.325116 10432 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:00.255535 17819 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:00.223372 17833 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:00.336979 10581 raft_consensus.cc:1275] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:00.323122 17830 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:13:00.339416 17766 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.343088 17820 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:00.333320 10430 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:00.333735 17778 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
W20260709 12:13:00.269557 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 95938.6us
W20260709 12:13:00.331718 17831 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:13:00.363380 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:00.365630 10433 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
I20260709 12:13:00.367419 10437 raft_consensus.cc:2468] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
W20260709 12:13:00.371613 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44196.6us
I20260709 12:13:00.373214 10431 raft_consensus.cc:2468] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 12.
I20260709 12:13:00.374573 10438 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:13:00.377848 17871 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:13:00.336100 17833 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:00.347652 17825 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.374022 17836 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:00.357451 17832 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:00.259202 17785 consensus_queue.cc:1175] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 22 status { last_received { term: 21 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:00.395140 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:00.379417 17874 ts_tablet_manager.cc:933] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:00.400507 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.400524 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:00.419421 10583 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 16: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:00.380729 17830 consensus_queue.cc:1048] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 12:13:00.403865 10578 raft_consensus.cc:1240] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
W20260709 12:13:00.413391 10550 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.375435 10478 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:00.328017 17745 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.431674 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:00.415210 17782 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.414523 17833 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:00.442379 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:00.435552 17825 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:00.421176 17832 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.656134 17832 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:00.439788 17782 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:00.528581 17871 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:13:00.437973 17823 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:13:00.644547 17827 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.616s	user 0.015s	sys 0.003s
I20260709 12:13:00.440006 17793 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:00.645035 17836 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:13:00.646080 17819 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:00.488783 10435 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:00.501566 10578 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:13:00.653623 17886 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.213s	user 0.005s	sys 0.000s
W20260709 12:13:00.653579 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 211351us
I20260709 12:13:00.686146 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:00.644091 17787 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.733s	user 0.015s	sys 0.008s
W20260709 12:13:00.688066 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 290424us
I20260709 12:13:00.658932 17782 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:00.667491 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 208837us
I20260709 12:13:00.690352 10553 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:00.669032 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 292) took 1117 ms. Trace:
I20260709 12:13:00.692368 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:00.678668 17874 tablet_copy_client.cc:323] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:13:00.661466 17777 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:13:00.696749 10578 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:13:00.677158 17848 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.704039 10553 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:00.701856 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.691514 10335 rpcz_store.cc:276] 0709 12:12:59.551908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.560175 (+  8267us) service_pool.cc:224] Handling call
0709 12:13:00.668957 (+1108782us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:59.838532 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:59.936648 (+ 98116us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:59.938249 (+  1601us) write_op.cc:435] Acquired schema lock
0709 12:12:59.939505 (+  1256us) tablet.cc:662] Decoding operations
0709 12:13:00.015962 (+ 76457us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:00.023069 (+  7107us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:00.023113 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:00.023365 (+   252us) tablet.cc:701] Row locks acquired
0709 12:13:00.023388 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:13:00.023564 (+   176us) write_op.cc:270] Start()
0709 12:13:00.023718 (+   154us) write_op.cc:276] Timestamp: P: 1783599180023513 usec, L: 0
0709 12:13:00.023743 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:13:00.024618 (+   875us) log.cc:844] Serialized 535 byte log entry
0709 12:13:00.362723 (+338105us) op_driver.cc:464] REPLICATION: finished
0709 12:13:00.649608 (+286885us) write_op.cc:301] APPLY: starting
0709 12:13:00.649769 (+   161us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:00.651127 (+  1358us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:00.651234 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:00.652223 (+   989us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:00.652443 (+   220us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:13:00.652495 (+    52us) write_op.cc:312] APPLY: finished
0709 12:13:00.652734 (+   239us) log.cc:844] Serialized 25 byte log entry
0709 12:13:00.657601 (+  4867us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.657700 (+    99us) write_op.cc:454] Released schema lock
0709 12:13:00.657744 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":280609,"apply.run_cpu_time_us":4060,"apply.run_wall_time_us":31859,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":203133,"prepare.run_cpu_time_us":4468,"prepare.run_wall_time_us":236761,"replication_time_us":312159,"thread_start_us":159053,"threads_started":2}]]}
I20260709 12:13:00.698046 17832 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:00.688093 17823 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:00.701866 17787 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:13:00.698307 17777 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:00.701143 17836 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:13:00.700927 17793 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:00.706965 17847 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:00.745148 17894 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:13:00.707414 17766 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:00.693495 17833 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.736621 17883 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:13:00.750229 17745 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.707754 17848 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.786278 17777 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.787606 17777 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:00.766126 17836 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:00.803270 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 428022us
W20260709 12:13:00.804088 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.765928 17766 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.808027 11213 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:13:00.815956 17900 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:00.816776 17822 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:00.816632 17900 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.817363 17822 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.818974 17900 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:00.829947 10504 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:13:00.842728 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:00.843775 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:00.843780 10509 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:00.844748 10506 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
W20260709 12:13:00.850538 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44947.5us
I20260709 12:13:00.809307 17807 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:00.856073 10505 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:00.800662 17855 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:13:00.863118 10510 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:13:00.865763 17807 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:00.798079 17745 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.803418 17836 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:00.881168 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 80038us
I20260709 12:13:00.837239 17820 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:00.849623 17819 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:13:00.889029 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 88586.5us
I20260709 12:13:00.891382 17879 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:00.892261 17879 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.894366 11216 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:13:00.895685 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet c545de01b7fb4c89946871c1d522006f from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:13:00.896274 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet c545de01b7fb4c89946871c1d522006f from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-c545de01b7fb4c89946871c1d522006f
I20260709 12:13:00.901899 17895 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:00.905751 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:00.907254 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:00.909117 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:00.909816 10431 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:00.913923 10434 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:00.858546 17887 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:13:00.812083 17823 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:00.921415 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:00.921756 17833 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:00.922494 10510 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:00.922672 17745 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:00.899531 17820 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.882709 17807 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.936777 17848 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:00.811762 17793 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.903272 17847 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:00.929726 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 78074.7us
I20260709 12:13:00.914770 17822 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:00.907738 17819 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.952941 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:00.954309 10583 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:13:00.915479 17782 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:13:01.067562 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:13:01.045918 17831 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:13:01.094161 10510 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:13:01.066771 17843 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:00.951838 17847 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:00.951066 17901 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.142s
W20260709 12:13:01.092409 17889 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:13:01.079588 17879 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.087203 17564 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.108137 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 55299.6us
I20260709 12:13:01.110415 17820 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:01.116025 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48608.7us
I20260709 12:13:01.109196 17843 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.120363 17804 consensus_queue.cc:1059] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:13:01.127792 17843 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:01.128031 10597 tablet_copy_source_session.cc:215] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:01.130649 10436 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:01.112051 17909 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.116604 17766 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.146569 17836 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.142271 17912 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:13:01.117473 17564 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.193740 10436 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:01.199923 17847 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.200654 17564 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.185261 17912 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:01.187772 17830 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:01.142566 17909 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.219861 17912 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.202805 17819 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.227735 17909 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.228276 17832 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.228943 17832 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.233552 17913 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:01.234488 17909 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.235139 17909 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.234833 10328 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 171) took 1274 ms. Trace:
I20260709 12:13:01.234491 17913 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:01.237037 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 287736us
I20260709 12:13:01.257110 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.257932 10435 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:01.259099 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.240836 17832 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.259825 10438 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:13:01.259938 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.260656 10432 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:01.208359 17830 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.262174 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.263548 10434 raft_consensus.cc:1275] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:13:01.265563 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.236868 10328 rpcz_store.cc:276] 0709 12:12:59.960599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.960776 (+   177us) service_pool.cc:224] Handling call
0709 12:13:01.234749 (+1273973us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:00.108813 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:00.109577 (+   764us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:00.109631 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:13:00.109660 (+    29us) tablet.cc:662] Decoding operations
0709 12:13:00.110198 (+   538us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:00.110279 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:00.110325 (+    46us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:00.110592 (+   267us) tablet.cc:701] Row locks acquired
0709 12:13:00.110620 (+    28us) write_op.cc:260] PREPARE: finished
0709 12:13:00.110845 (+   225us) write_op.cc:270] Start()
0709 12:13:00.111045 (+   200us) write_op.cc:276] Timestamp: P: 1783599180110780 usec, L: 0
0709 12:13:00.111079 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:13:00.112032 (+   953us) log.cc:844] Serialized 530 byte log entry
0709 12:13:00.761557 (+649525us) op_driver.cc:464] REPLICATION: finished
0709 12:13:00.806033 (+ 44476us) write_op.cc:301] APPLY: starting
0709 12:13:00.807175 (+  1142us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:00.890667 (+ 83492us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:00.890794 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:00.891743 (+   949us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:00.891916 (+   173us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:13:00.891976 (+    60us) write_op.cc:312] APPLY: finished
0709 12:13:00.892177 (+   201us) log.cc:844] Serialized 25 byte log entry
0709 12:13:00.892372 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.892460 (+    88us) write_op.cc:454] Released schema lock
0709 12:13:00.892498 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:00.892972 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.892994 (+    22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":8076,"apply.run_cpu_time_us":4481,"apply.run_wall_time_us":93466,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":35062,"prepare.run_cpu_time_us":3574,"prepare.run_wall_time_us":3573,"replication_time_us":645551}]]}
I20260709 12:13:01.273638 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.275121 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:01.275388 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.278260 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.279659 10582 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:13:01.282151 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:01.276465 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:01.285332 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:01.236577 17920 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.270759 17830 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:01.285243 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:01.306190 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:01.307077 11220 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
W20260709 12:13:01.307670 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 320055us
I20260709 12:13:01.307896 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.308658 10580 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:01.309978 11218 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:13:01.310132 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.240988 17874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c545de01b7fb4c89946871c1d522006f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:01.263005 17913 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.251071 17909 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.313617 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.310865 10584 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
I20260709 12:13:01.317229 10505 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:01.281855 17847 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.284430 17778 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:01.316844 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:01.318395 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.283541 17885 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:13:01.320834 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.324345 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.324774 10510 raft_consensus.cc:2393] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:01.320996 17847 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.314028 17887 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:01.327786 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.319841 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.261547 17799 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:13:01.318311 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.325237 10508 raft_consensus.cc:2393] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 23: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:01.307144 17843 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:01.325897 11177 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:01.330693 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.330314 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.369515 10582 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:01.325059 17778 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.318145 17920 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.372104 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.368564 10585 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:01.319160 10553 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:01.375607 10585 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:01.329185 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.377764 10582 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:13:01.373893 17920 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.375869 10506 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:01.318074 17922 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.398042 10579 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
W20260709 12:13:01.401051 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 82657.9us
I20260709 12:13:01.369057 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:01.369700 17843 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.405223 17843 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:01.411306 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 91188.7us
I20260709 12:13:01.411620 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:01.415771 17820 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:01.416229 17820 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:01.416599 17820 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:01.420742 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:01.421365 10505 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:13:01.422668 17879 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:01.423147 17879 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.423511 17879 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:01.431845 17906 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:01.434522 17911 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:01.437575 17843 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.440812 17843 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.448333 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:01.450171 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:01.456072 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87131.1us
I20260709 12:13:01.358701 17785 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:01.460702 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.319128 17923 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:01.330893 17887 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.399027 17778 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:01.328088 17905 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:01.325222 17766 consensus_queue.cc:1048] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 12:13:01.397691 17848 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:01.469712 17887 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:01.465644 17923 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.495802 17905 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.458164 17836 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:01.523589 17836 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:01.524317 17836 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:01.454664 17922 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.500269 17848 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:01.531603 17848 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:01.466969 17872 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:01.532747 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.533555 10586 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:13:01.515879 17924 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:01.522212 17816 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.462393 17920 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:01.534776 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:01.459162 17777 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 12:13:01.460541 17833 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:13:01.489245 17804 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:01.571244 17920 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:01.582005 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47342.4us
I20260709 12:13:01.510191 17909 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:01.587718 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 137649us
I20260709 12:13:01.512545 17923 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:01.594295 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.594620 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.590148 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.596684 10437 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:13:01.596823 10434 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
I20260709 12:13:01.567845 17782 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:01.588253 17920 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:01.601270 17915 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.588940 17804 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.602180 17915 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.604766 17804 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:01.540694 17913 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.607947 17913 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.601084 17782 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.622798 17782 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:01.539430 17922 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.568733 17816 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.627462 10578 raft_consensus.cc:2393] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:01.598181 17909 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.632503 17905 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:01.632725 17847 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.610634 17912 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:01.576151 17895 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.637756 17895 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.644897 17816 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.641880 17913 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.651074 17909 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:01.653566 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57368.8us
I20260709 12:13:01.664594 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:01.626547 17922 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.672673 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79560.8us
I20260709 12:13:01.641495 17895 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:01.671605 10436 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 4301) took 1401 ms (client timeout 1583 ms). Trace:
W20260709 12:13:01.683379 10436 rpcz_store.cc:269] 0709 12:13:00.262105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.262613 (+   508us) service_pool.cc:224] Handling call
0709 12:13:01.663663 (+1401050us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54002652d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:01.663812 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1837743488}
I20260709 12:13:01.689716 17847 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.690869 10552 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:01.690662 17847 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.692922 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:01.694485 10552 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:01.695242 17782 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:01.713346 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58295us
W20260709 12:13:01.718672 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45122.2us
I20260709 12:13:01.694208 17847 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:01.714407 10552 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:01.681881 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 92630.7us
I20260709 12:13:01.689623 17848 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.747844 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:01.747880 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:13:01.749009 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 154111us
I20260709 12:13:01.749568 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.752255 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.761052 10506 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:13:01.761615 10507 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:01.715891 17777 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:01.777557 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:01.780920 11217 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:13:01.721354 17564 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.726768 17832 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:01.782642 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.694849 17879 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:01.788661 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74395.9us
I20260709 12:13:01.789042 10513 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:01.743288 17905 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.727972 17882 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.751797 17799 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.799499 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.745116 17909 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.804415 10437 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:01.786962 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:01.787835 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:01.679737 17922 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.795295 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.816004 10434 raft_consensus.cc:2393] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:01.796449 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:13:01.797623 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 50169.4us
I20260709 12:13:01.759384 17822 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.799199 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 44751.2us
I20260709 12:13:01.790304 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:13:01.840960 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 50759.7us
I20260709 12:13:01.802634 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.737142 17944 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:13:01.769289 17819 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:01.807780 10506 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
I20260709 12:13:01.810374 17954 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.811463 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.812117 17955 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:01.819777 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.842904 10431 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:01.775452 17820 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.855531 11214 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:01.872845 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.873848 10506 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:13:01.790978 17564 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:01.880087 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 132345us
I20260709 12:13:01.876315 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:01.846568 17944 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.852223 17832 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:01.851956 17955 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:01.802170 17905 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.888229 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.852088 17913 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.856098 17901 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.803895 17882 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.821862 17799 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:01.898874 10584 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:13:01.901064 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.875836 17872 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.908141 11220 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:01.858516 17839 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.821346 17909 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.907590 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.908522 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:01.855772 17954 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.891464 17955 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:01.922016 17848 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.918498 17878 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:01.952780 17878 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.910393 17915 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:01.935525 17925 log.cc:927] Time spent T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.473s	user 0.006s	sys 0.000s
I20260709 12:13:01.956492 17878 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.955874 17839 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.915524 10436 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 20: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:01.935245 17939 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:01.885332 17944 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:01.958534 17939 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:01.958952 17939 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:01.918249 17907 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.925087 17778 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.960673 17799 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.961436 10583 raft_consensus.cc:2368] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
W20260709 12:13:01.937366 17947 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:13:01.961854 17847 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:01.942505 10432 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:01.925860 17564 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.884202 17832 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.963953 17832 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:01.966056 17905 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:01.891249 17941 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:01.967382 17941 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:01.968266 17836 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.852137 17819 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.896827 17913 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.972609 17819 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:01.849558 17822 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.962729 17847 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:13:01.977355 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60963.2us
I20260709 12:13:01.954939 17909 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:01.996026 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 142065us
I20260709 12:13:01.947471 17820 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:01.997941 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 149693us
I20260709 12:13:02.001037 10435 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:13:02.002209 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:02.003962 10433 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:13:02.004511 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.005060 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.003854 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.006299 10429 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:02.006654 10435 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:13:02.007290 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:02.007356 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.008188 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:02.008363 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:02.002478 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:02.015681 10431 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:02.021912 17913 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.022835 17913 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:02.023550 17958 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.024936 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:02.025657 11220 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:13:02.025892 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:01.928081 17879 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.026875 10435 raft_consensus.cc:2393] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:02.026151 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.029121 10507 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 15: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:01.929893 17922 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.024353 17958 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:01.941483 17882 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.975420 17941 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.032963 10431 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
W20260709 12:13:02.035142 10479 consensus_peers.cc:597] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 945dd7b821a5493f8a9d6ca251795da9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:01.904609 17901 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.038753 17941 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:02.048899 17901 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:01.945397 17872 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.054123 17822 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.055346 10511 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:02.048676 17941 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.056093 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:02.059007 17920 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.060039 17923 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.065203 17958 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.065567 17874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57892 (request call id 1108) took 2135 ms. Trace:
I20260709 12:13:02.056098 17872 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.049639 17820 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:02.052177 17847 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.066190 17874 rpcz_store.cc:276] 0709 12:12:59.929832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.930017 (+   185us) service_pool.cc:224] Handling call
0709 12:13:00.422234 (+492217us) ts_tablet_manager.cc:934] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
0709 12:13:01.141531 (+719297us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:01.729606 (+588075us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:01.729890 (+   284us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:02.064108 (+334218us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:02.064739 (+   631us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:02.065303 (+   564us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:02.065403 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":1652,"tablet-copy.queue_time_us":327337,"thread_start_us":271809,"threads_started":2}
I20260709 12:13:02.088462 11215 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:02.044119 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.070329 17832 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.071539 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.072995 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:02.099643 10584 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:13:02.101413 17914 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:02.102084 17914 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.102196 17792 raft_consensus.cc:1064] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:13:02.102942 17914 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:02.105518 17832 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:02.053218 17830 consensus_meta.cc:287] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.734s	user 0.008s	sys 0.008s
I20260709 12:13:02.131626 17830 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.077838 17954 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.078423 17839 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.058658 17941 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.079058 17778 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.080811 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:02.083676 17887 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.011s
I20260709 12:13:01.992103 17907 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.085153 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:02.087705 17909 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:02.106297 17923 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.062574 17920 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.112303 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:02.111893 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:02.129868 11215 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:13:02.214937 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 103207us
I20260709 12:13:02.140673 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:01.311784  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:02.163932 10335 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 11, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:13:02.191815 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 73173.3us
I20260709 12:13:02.214496 10583 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:02.191943 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46763.7us
W20260709 12:13:02.206691 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48425.9us
W20260709 12:13:02.210362 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 125436us
I20260709 12:13:02.218395 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:02.222723 10506 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 23.
I20260709 12:13:02.223315 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.224174 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:02.224174 10504 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:13:02.198577 17939 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.216428 10584 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:02.268565 17951 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:02.268596 10478 leader_election.cc:400] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:02.270954 10478 leader_election.cc:403] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:02.257652 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.214212 17807 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:02.259143 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.261169 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:02.277571 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.315176 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:02.225059 10510 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:13:02.316403 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56450us
W20260709 12:13:02.261332 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 43642.5us
I20260709 12:13:02.264590 10583 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:13:02.262912 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.218466 17948 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:13:02.315675 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 47377us
I20260709 12:13:02.318094 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.217674 17874 tablet_copy_client.cc:806] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:02.276654 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:02.322149 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:02.322430 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.318706 11177 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:02.325248 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:02.326118 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:02.326884 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:02.327622 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:02.357326 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:02.373469 10508 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:02.342883 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:02.374747 10586 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:02.357993 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.380076 11215 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:13:02.331483 10335 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 18, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:02.339743 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:02.272020 17777 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:02.378575 11216 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:02.271476 17836 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.224249 17872 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:02.396111 10579 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:02.209991 17887 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:02.375082 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 98528.1us
W20260709 12:13:02.376569 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 58012.1us
I20260709 12:13:02.341931 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 312) took 2530 ms. Trace:
I20260709 12:13:02.357985 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.377602 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.316478 17847 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:02.339784 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.214387 17909 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.271051 17920 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:02.388373 10587 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:02.389736 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.404663 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.413784 10512 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:13:02.414422 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.316293 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:02.415445 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.347078 17564 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:13:02.226042 17961 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.160s
I20260709 12:13:02.412014 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:02.416685 11216 raft_consensus.cc:2368] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 23. Current term is 24.
I20260709 12:13:02.417451 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.363936 17870 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:02.415139 10582 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:13:02.423157 11177 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:02.375765 17879 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:02.346766 17830 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.428500 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:02.341423 17819 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:02.398746 17793 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.376991 17948 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.442323 17948 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:02.429232 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.400808 10336 rpcz_store.cc:276] 0709 12:12:59.795779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.837979 (+ 42200us) service_pool.cc:224] Handling call
0709 12:13:02.325768 (+2487789us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:00.025170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:00.025750 (+   580us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:00.027707 (+  1957us) write_op.cc:435] Acquired schema lock
0709 12:13:00.027732 (+    25us) tablet.cc:662] Decoding operations
0709 12:13:00.028251 (+   519us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:00.028300 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:00.028327 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:00.028533 (+   206us) tablet.cc:701] Row locks acquired
0709 12:13:00.028552 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:13:00.028722 (+   170us) write_op.cc:270] Start()
0709 12:13:00.028837 (+   115us) write_op.cc:276] Timestamp: P: 1783599180028646 usec, L: 0
0709 12:13:00.028853 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:13:00.037092 (+  8239us) log.cc:844] Serialized 535 byte log entry
0709 12:13:00.647107 (+610015us) op_driver.cc:464] REPLICATION: finished
0709 12:13:00.739769 (+ 92662us) write_op.cc:301] APPLY: starting
0709 12:13:00.743724 (+  3955us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:00.882595 (+138871us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:00.914665 (+ 32070us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:00.922376 (+  7711us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:00.922570 (+   194us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:13:00.922629 (+    59us) write_op.cc:312] APPLY: finished
0709 12:13:00.922867 (+   238us) log.cc:844] Serialized 25 byte log entry
0709 12:13:00.923126 (+   259us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.923226 (+   100us) write_op.cc:454] Released schema lock
0709 12:13:00.923270 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:00.923701 (+   431us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:00.923732 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":303035,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":61595,"apply.run_cpu_time_us":4862,"apply.run_wall_time_us":189900,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":132680,"prepare.run_cpu_time_us":2790,"prepare.run_wall_time_us":12254,"replication_time_us":618055,"thread_start_us":82817,"threads_started":1}]]}
W20260709 12:13:02.448688 17967 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:13:02.431463 10580 raft_consensus.cc:2368] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:02.434127 17848 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:02.434315 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:02.450243 17963 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:13:02.450368 17848 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:02.434667 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.452500 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.452006 17848 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.456540 17931 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:13:02.458436 17792 consensus_queue.cc:237] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.460661 17778 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:02.462792 17908 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:02.399636 17874 tablet_copy_client.cc:670] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:02.387784 17965 raft_consensus.cc:2804] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:02.428251 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.428983 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.441725 17938 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:02.447527 17903 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:02.436311 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:02.492854 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.493127 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.493782 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:02.493871 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:02.495512 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.492260 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.420790 17872 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:02.511353 17872 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.489374 17836 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:13:02.498364 10582 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:13:02.515367 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 79177.3us
I20260709 12:13:02.508211 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:02.499326 10511 raft_consensus.cc:2368] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20.
I20260709 12:13:02.507972 10434 raft_consensus.cc:3060] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:13:02.436424 10587 raft_consensus.cc:3060] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:02.509160 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:02.469933 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:02.509922 10430 raft_consensus.cc:3060] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:02.519827 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49958us
I20260709 12:13:02.472116 10508 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:02.526731 10553 leader_election.cc:400] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 23. Current term is 24.
I20260709 12:13:02.527343 10553 leader_election.cc:403] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:13:02.514549 17926 log.cc:927] Time spent T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:13:02.434362 17944 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.534186 17778 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:02.535306 11213 raft_consensus.cc:2368] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:02.539265 17830 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:02.421859 17847 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.546746 17778 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:02.548482 17778 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.524461 17965 raft_consensus.cc:697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:02.552748 17965 consensus_queue.cc:237] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.556109 17939 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.553525 10580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 4410) took 1227 ms. Trace:
I20260709 12:13:02.559440 10580 rpcz_store.cc:276] 0709 12:13:01.326035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.326176 (+   141us) service_pool.cc:224] Handling call
0709 12:13:02.553182 (+1227006us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001caf54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:02.553397 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434032512}
I20260709 12:13:02.561818 17948 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.501938 17825 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:02.565647 10585 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
W20260709 12:13:02.567185 17970 log.cc:927] Time spent T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
I20260709 12:13:02.568719 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.570276 10579 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
W20260709 12:13:02.575389 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48876.8us
I20260709 12:13:02.619871 10584 raft_consensus.cc:2468] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 24.
I20260709 12:13:02.579974 10580 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:13:02.622354 17955 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.008s
I20260709 12:13:02.596117 10434 raft_consensus.cc:2468] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:13:02.468109 17819 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:02.581562 10430 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:13:02.427827 17920 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:02.599718 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 91602.9us
I20260709 12:13:02.599773 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.600365 17839 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:02.515219 17903 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:02.632634 11220 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:02.632361 10552 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.439661 17879 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.492884 17793 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.654109 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.663959 10553 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:02.632412 17905 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.665553 10513 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:13:02.673702 10552 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.666915 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:02.465813 17870 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.678311 11217 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:02.639065 17847 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.685824 17895 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:02.682219 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.688365 10506 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:13:02.623871 17825 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:02.691285 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.692333 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.653286 17839 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.658636 17903 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:02.698673 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:02.675750 17836 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:02.711505 17937 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:02.704785 17839 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:02.636524 17955 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:02.712165 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.690665 17946 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 23. Current term is 24.
W20260709 12:13:02.715134 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48345.9us
I20260709 12:13:02.681556 17883 raft_consensus.cc:2804] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:02.741809 10504 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:02.683789 17905 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.751546 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.748974 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.753034 17820 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:02.753477 17820 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.753846 17820 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:02.743886 10587 raft_consensus.cc:2468] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:02.756932 17922 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:02.758438 10479 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:02.762169 17789 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:02.762584 17789 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.762997 17789 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:02.753038 11178 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.768553 11178 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:02.769245 17922 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:02.770366 17922 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:02.775435 17564 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:02.775748 10583 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:13:02.775990 17564 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.736335 17825 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.743800 17883 raft_consensus.cc:697] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:13:02.783334 17816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.009s	sys 0.019s
W20260709 12:13:02.807222 17816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.009s	sys 0.020s
I20260709 12:13:02.809598 17793 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.811446 17819 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:13:02.787396 17887 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.785924 17911 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:02.814545 17911 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.816764 17920 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.818796 17955 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.807551 17564 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.848351 11177 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:02.855103 10586 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:02.856297 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:02.858414 10582 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:02.805958 17909 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:02.874747 11177 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:02.826912 17911 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.832286 17905 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.833447 17907 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.833065 17870 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.879159 17920 raft_consensus.cc:2804] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:02.880986 17909 raft_consensus.cc:2804] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:02.882532 17879 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:02.920686 17874 tablet_copy_client.cc:538] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:02.907158 17909 raft_consensus.cc:697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:02.924654 17909 consensus_queue.cc:237] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.834955 17883 consensus_queue.cc:237] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:02.963202 17920 raft_consensus.cc:697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:02.965422 17920 consensus_queue.cc:237] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:02.969535 17948 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:02.989385 17944 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:02.998538 17914 consensus_meta.cc:287] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.895s	user 0.015s	sys 0.001s
I20260709 12:13:03.023767 17914 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.028283 17914 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:03.056999 17938 consensus_meta.cc:287] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.012s
I20260709 12:13:03.057914 17938 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
W20260709 12:13:03.109328 10335 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.017s	sys 0.041s
W20260709 12:13:03.110517 10335 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.018s	sys 0.041s
I20260709 12:13:03.113883 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:03.115876 10429 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.116444 10403 leader_election.cc:400] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20.
I20260709 12:13:03.117079 10403 leader_election.cc:403] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:03.121227 10404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.000s	sys 0.023s
W20260709 12:13:03.122021 10404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.000s	sys 0.023s
I20260709 12:13:03.124048 10404 leader_election.cc:400] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:03.126143 10431 raft_consensus.cc:2393] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
W20260709 12:13:03.127381 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.127434 10404 leader_election.cc:403] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:03.135591 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 344870us
I20260709 12:13:03.156131 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:03.122273 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 392743us
I20260709 12:13:02.887292 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:03.172664 10510 raft_consensus.cc:2468] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
W20260709 12:13:03.174002 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 286640us
I20260709 12:13:03.178038 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.178303 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:03.181310 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.181844 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.183854 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:03.183866 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:03.184839 10585 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:03.185526 11212 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.186583 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:03.198699 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.199059 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.205421 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.205865 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.207789 17793 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.225153 17843 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:03.225852 17799 raft_consensus.cc:3060] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:13:03.226567 17843 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.234455 17843 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:03.234912 17792 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:03.236109 17792 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.236750 17830 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:03.237304 17830 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.237830 17830 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:03.239574 17879 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:03.240810 17792 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:03.240630 17879 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.241961 17792 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:03.251175 10580 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:13:03.254196 17848 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:13:03.255280 17839 consensus_meta.cc:287] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.005s
I20260709 12:13:03.255971 17839 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.256304 17879 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.286121 10429 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:13:03.287767 10585 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:13:03.284416 17839 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.300040 17822 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:03.303367 10552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.001s	sys 0.012s
W20260709 12:13:03.303599 17820 consensus_meta.cc:287] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.012s
W20260709 12:13:03.304337 17789 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.007s
I20260709 12:13:03.304288 17820 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.305074 17789 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.307839 17820 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:03.308780 10552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.005s	sys 0.012s
I20260709 12:13:03.309331 17906 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:13:03.310453 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 636848us
I20260709 12:13:03.311092 17879 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.311726 17807 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.311997 17879 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.312777 17807 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:03.315680 17879 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.316857 17985 ts_tablet_manager.cc:933] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:03.331693 11212 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:03.336068 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:03.310276 17906 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.361653 11177 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:03.302881 17822 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.375663 17807 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:03.376407 10513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.003s	sys 0.006s
I20260709 12:13:03.391745 10431 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:13:03.375842 10437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.010s	sys 0.003s
I20260709 12:13:03.374409 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.372365 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:03.403576 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 59157.1us
W20260709 12:13:03.416597 10437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.010s	sys 0.004s
I20260709 12:13:03.415760 17789 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:03.438336 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 100444us
I20260709 12:13:03.443634 10404 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:03.448309 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.451390 17978 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:03.455618 17965 raft_consensus.cc:2804] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:13:03.456669 17965 raft_consensus.cc:697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:03.458273 17906 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.463493 17870 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:03.464407 17870 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:13:03.462034 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48463.7us
W20260709 12:13:03.465351 10504 consensus_meta.cc:287] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.717s	user 0.014s	sys 0.012s
I20260709 12:13:03.471127 10552 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:03.473328 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 183704us
I20260709 12:13:03.474278 11218 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:03.466488 17793 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20.
I20260709 12:13:03.477334 10435 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:03.478238 10585 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:13:03.478381 10435 raft_consensus.cc:606] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:03.483636 17895 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:03.483803 17822 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:03.480810 10504 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:13:03.484470 17895 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:03.492225 17895 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.493315 17939 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:03.493932 17939 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:03.494398 17939 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:03.470137 17985 tablet_copy_client.cc:323] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:13:03.500795 17920 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:03.501678 10507 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38166 (request call id 1109) took 1281 ms. Trace:
I20260709 12:13:03.511451 17870 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.515405 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:03.502128 10507 rpcz_store.cc:276] 0709 12:13:02.220107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.221400 (+  1293us) service_pool.cc:224] Handling call
0709 12:13:03.501284 (+1279884us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540038f4d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:03.501470 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2698490624}
I20260709 12:13:03.522594 17822 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.523562 17822 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.526491 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:03.528947 17822 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.528051 10509 raft_consensus.cc:1240] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:03.458438 17965 consensus_queue.cc:237] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.528245 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:03.533063 10506 raft_consensus.cc:2393] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:03.525799 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:03.528224 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 945dd7b821a5493f8a9d6ca251795da9 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:13:03.535669 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 945dd7b821a5493f8a9d6ca251795da9 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-945dd7b821a5493f8a9d6ca251795da9
I20260709 12:13:03.535682 10507 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:03.542519 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:03.506080 17799 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
W20260709 12:13:03.554865 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47215.2us
I20260709 12:13:03.557991 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.559283 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.559473 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.561232 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:03.563760 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:03.553349 10504 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 3824) took 1228 ms (client timeout 1586 ms). Trace:
W20260709 12:13:03.567154 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 132598us
I20260709 12:13:03.567162 17941 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.569350 17941 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:03.556169 17793 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:03.564759 10430 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:03.571972 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:03.573423 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.573854 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:03.575189 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.575904 10431 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:03.578897 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:03.580555 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.580266 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.582975 10431 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:13:03.584340 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:03.589151 17941 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:03.595762 17903 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.882s	user 0.011s	sys 0.010s
I20260709 12:13:03.596382 17903 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:03.566123 10504 rpcz_store.cc:269] 0709 12:13:02.322726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.339468 (+ 16742us) service_pool.cc:224] Handling call
0709 12:13:03.538831 (+1199363us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54003165d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:03.550946 (+ 12115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827715328}
W20260709 12:13:03.594769 17997 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:13:03.613389 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:03.600625 17978 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:03.624130 17874 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:13:03.627053 10578 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 16.
W20260709 12:13:03.628659 10578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55480 (request call id 4438) took 1288 ms (client timeout 1627 ms). Trace:
I20260709 12:13:03.628705 10437 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 3320) took 1135 ms. Trace:
W20260709 12:13:03.629153 10578 rpcz_store.cc:269] 0709 12:13:02.340596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.377290 (+ 36694us) service_pool.cc:224] Handling call
0709 12:13:03.628391 (+1251101us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400208fd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:03.628564 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2746029440}
I20260709 12:13:03.657397 17948 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.657708 17922 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.658257 17948 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.658692 17922 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.661163 17948 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.672981 10582 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:13:03.629356 10437 rpcz_store.cc:276] 0709 12:13:02.493718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.494386 (+   668us) service_pool.cc:224] Handling call
0709 12:13:02.494788 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.496863 (+  2075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:02.496895 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:02.574627 (+ 77732us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b18001bef80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9c02a6 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a
0709 12:13:02.612071 (+ 37444us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b2c00000f48. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c913d8a11 00007f2c913e8372 00007f2c92b1aa01 00007f2c92adf43a 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2
0709 12:13:02.634832 (+ 22761us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.454706 (+819874us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:03.601988 (+147282us) log.cc:844] Serialized 14 byte log entry
0709 12:13:03.617627 (+ 15639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.628345 (+ 10718us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:03.628415 (+    70us) raft_consensus.cc:1685] finished
0709 12:13:03.628431 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:03.628601 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31541,"raft.queue_time_us":21332,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":21548,"spinlock_wait_cycles":11931264,"thread_start_us":861496,"threads_started":2,"wal-append.queue_time_us":790289}
I20260709 12:13:03.651834 17939 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:03.661854 17922 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:03.687098 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47420.3us
I20260709 12:13:03.688390 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.690181 10479 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:03.691265 17822 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:03.691711 17822 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.692071 17822 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:03.693063 17939 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:03.670218 17830 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.693945 17879 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:03.690476 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:03.705945 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 58303us
I20260709 12:13:03.622259 17793 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 12:13:03.708186 17830 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:03.717506 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:03.707317 17948 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:13:03.718315 10504 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.720963 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:03.726558 10478 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:03.728459 17903 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:03.731828 17879 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:03.741324 10504 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:13:03.740960 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:03.744561 10582 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:03.750799 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:03.753960 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:03.756388 11213 raft_consensus.cc:2368] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:13:03.762467 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:03.775192 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:03.767911 17948 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:03.781543 17948 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:03.782895 17948 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.787086 17879 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:03.794687 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:03.795849 17879 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:03.797868 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:03.798321 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.800050 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.801884 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.803570 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:03.804553 10585 raft_consensus.cc:2368] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:03.768673 17903 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.809393 17879 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:13:03.817453 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.817749 17903 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.811019 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:03.811703 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:03.813167 10586 raft_consensus.cc:1275] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:03.816035 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:03.813076 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:03.794898 10507 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
W20260709 12:13:03.848241 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 53648.6us
I20260709 12:13:03.852648 17822 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.865366 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.865392 17792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:38164 (request call id 16) took 3657 ms. Trace:
I20260709 12:13:03.865525 17789 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:03.869457 10434 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.873466 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.875216 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.876065 10587 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:13:03.877332 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.872854 10583 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 12.
W20260709 12:13:03.873014 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71991.2us
I20260709 12:13:03.859458 17879 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.871513 17789 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.871285 17820 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.890161 17789 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.890378 17872 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:13:03.863969 17939 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:03.907680 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 96847us
I20260709 12:13:03.906579 17988 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:03.911526 17820 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.912138 17988 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.914528 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:13:03.919855 18009 log.cc:927] Time spent T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:13:03.926851 17939 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.929639 17879 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.931423 10435 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:03.929874 17789 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:03.941515 10434 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
W20260709 12:13:03.944489 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 63338.4us
I20260709 12:13:03.945138 18019 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:03.945422 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:03.945854 18019 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:03.948457 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:03.951197 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:03.952348 10504 raft_consensus.cc:2368] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:13:03.940543 10431 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:03.869400 17792 rpcz_store.cc:276] 0709 12:13:00.211735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.212116 (+   381us) service_pool.cc:224] Handling call
0709 12:13:00.232972 (+ 20856us) log.cc:844] Serialized 492 byte log entry
0709 12:13:00.799089 (+566117us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b1800323a40. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c92a38b0b 00007f2c92abdb63 00007f2c92ae5752 00007f2c97aa5d29
0709 12:13:01.099488 (+300399us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b5400144c20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a5fb26 00007f2c92a5949a 00007f2c92a58714 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634
0709 12:13:01.185836 (+ 86348us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b5400144c20. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:13:03.861781 (+2675945us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12984,"raft.queue_time_us":400482,"raft.run_cpu_time_us":34230,"raft.run_wall_time_us":1611495,"spinlock_wait_cycles":12782720,"thread_start_us":173973,"threads_started":1,"wal-append.queue_time_us":183665,"wal-append.run_cpu_time_us":7508,"wal-append.run_wall_time_us":1356849}
I20260709 12:13:03.956426 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:03.956643 17792 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:03.959511 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.939999 17822 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:03.974022 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.971475 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:03.981695 10505 raft_consensus.cc:1240] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:13:03.975474 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:03.983687 10513 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 14.
I20260709 12:13:03.999739 18019 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:03.999948 17931 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:04.001216 10578 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 14: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:13:04.012683 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 64342us
W20260709 12:13:04.023976 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 161728us
W20260709 12:13:04.024856 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 79524.8us
I20260709 12:13:04.026602 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.027057 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:04.039748 17822 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:13:04.040719 10478 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.043099 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:04.044819 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:04.046257 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.047276 17792 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:04.047933 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.068986 10578 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:04.000945 17931 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:04.075314 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48727.5us
I20260709 12:13:04.076869 10479 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:04.081425 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.088097 17906 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:04.089759 17906 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:04.090250 10523 tablet_copy_source_session.cc:215] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:04.090292 17906 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:04.078596 18019 raft_consensus.cc:2764] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:13:04.079813 17792 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:13:04.096920 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50763.1us
I20260709 12:13:04.097004 18025 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.100008 18025 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.102238 10583 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:04.091874 17906 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.100466 18026 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:04.105933 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.093077 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:04.114674 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 97311.8us
I20260709 12:13:04.117918 17931 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.118897 18029 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:04.107465 17793 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:04.120395 17939 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.120256 17793 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.122143 17822 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:04.123333 10478 leader_election.cc:400] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:04.123698 10478 leader_election.cc:403] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:13:04.124367 17822 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:04.127597 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:04.102758 18025 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.116066 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:04.098358 17789 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.138553 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.140640 18030 consensus_queue.cc:1048] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:04.141027 17941 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:04.142959 17937 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.143613 17937 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.144179 17820 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.145241 17879 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.145975 17937 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.146471 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:04.111536 18028 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:04.122866 17793 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.156106 10552 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:04.147411 17792 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.159541 18025 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.160224 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:04.160763 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:04.172740 17941 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:04.172528 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.173108 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.173833 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.174333 17939 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.174855 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.128355 18026 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:04.175412 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:04.152299 17822 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:04.179499 10509 raft_consensus.cc:606] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:04.185035 18024 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:04.188300 10552 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:04.188665 10553 leader_election.cc:400] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:13:04.194736 10553 leader_election.cc:403] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:13:04.196346 10505 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:04.179739 17941 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.198320 10430 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:04.184031 17822 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:04.207186 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:04.218520 10384 auto_rebalancer.cc:629] Failed to schedule move for tablet f94d710c36f04a9b8685258a7ff9d8a0: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:13:04.189661 17948 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.219511 17948 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.203528 17941 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.222383 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.223122 17778 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.227087 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.185461 18025 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.214092 17822 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:04.234824 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:04.250404 18019 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.254191 18019 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.257476 18019 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:04.246618 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 55553us
I20260709 12:13:04.222394 17948 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:04.261999 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68782.2us
I20260709 12:13:04.233970 18033 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.267113 11217 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:04.262274 10435 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:13:04.209419 17939 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.273906 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.246423 18032 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.223827 17778 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:04.280478 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 76955.1us
I20260709 12:13:04.290079 10552 leader_election.cc:400] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:13:04.268486 17937 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:04.295548 10478 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.297485 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:04.298612 10510 raft_consensus.cc:2368] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
W20260709 12:13:04.300457 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 116591us
I20260709 12:13:04.304165 10478 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:04.307010 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:04.308364 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 126988us
I20260709 12:13:04.311856 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:04.314101 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.269554 18019 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.319375 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.277875 18032 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.279285 18034 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:04.287667 17883 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:04.327451 10504 raft_consensus.cc:3060] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:04.309760 18030 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:04.352872 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.315130 17872 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:13:04.358901 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:04.364913 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 54148.5us
I20260709 12:13:04.371976 10437 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:04.372946 10437 raft_consensus.cc:606] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:04.349582 17879 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:04.375660 10509 raft_consensus.cc:1275] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:13:04.380002 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73132.9us
I20260709 12:13:04.359438 17872 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.388118 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.320104 18033 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.390532 10508 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 12:13:04.391954 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 78351.7us
I20260709 12:13:04.393074 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:04.308254 18026 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.394869 10429 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 22.
I20260709 12:13:04.372236 18039 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.382946 17807 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.378847 17879 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.383118 17820 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:13:04.376868 17793 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:13:04.383797 18019 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.387193 17872 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:13:04.400799 17994 log.cc:927] Time spent T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.420s	user 0.000s	sys 0.006s
I20260709 12:13:04.371940 17909 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.166s
I20260709 12:13:04.397621 18038 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.410171 17948 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.427765 17879 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:13:04.442193 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 70831.1us
I20260709 12:13:04.456264  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:04.467325 18032 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:04.469609 18027 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:13:04.471436 17948 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:04.472607 18041 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:13:04.474150 17948 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.482007 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 301) took 3671 ms. Trace:
I20260709 12:13:04.432593 17807 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.493021 17807 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.483405  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:04.484401 10328 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 18 to 19, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:04.495347 18025 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.495536  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:04.438719 17820 consensus_queue.cc:1243] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:04.463956 10512 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:04.500473 18030 consensus_queue.cc:1059] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:04.505144 17778 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.507970 18019 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.493762 10335 rpcz_store.cc:276] 0709 12:13:00.810718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.810925 (+   207us) service_pool.cc:224] Handling call
0709 12:13:04.481870 (+3670945us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:02.943598 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:02.944812 (+  1214us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:02.944880 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:13:02.944920 (+    40us) tablet.cc:662] Decoding operations
0709 12:13:02.945724 (+   804us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:02.945842 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:02.945898 (+    56us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:13:02.946521 (+   623us) tablet.cc:701] Row locks acquired
0709 12:13:02.946562 (+    41us) write_op.cc:260] PREPARE: finished
0709 12:13:02.946832 (+   270us) write_op.cc:270] Start()
0709 12:13:03.029411 (+ 82579us) write_op.cc:276] Timestamp: P: 1783599182946742 usec, L: 0
0709 12:13:03.029466 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 12:13:03.030842 (+  1376us) log.cc:844] Serialized 968 byte log entry
0709 12:13:03.593931 (+563089us) op_driver.cc:464] REPLICATION: finished
0709 12:13:03.745193 (+151262us) write_op.cc:301] APPLY: starting
0709 12:13:03.747990 (+  2797us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:03.798015 (+ 50025us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:03.798149 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:03.800330 (+  2181us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:03.800705 (+   375us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:13:03.800780 (+    75us) write_op.cc:312] APPLY: finished
0709 12:13:03.801188 (+   408us) log.cc:844] Serialized 33 byte log entry
0709 12:13:03.803163 (+  1975us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:03.803345 (+   182us) write_op.cc:454] Released schema lock
0709 12:13:03.803410 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:03.870107 (+ 66697us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:03.871821 (+  1714us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":613426,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":140650,"apply.run_cpu_time_us":7133,"apply.run_wall_time_us":121893,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":541611,"prepare.run_cpu_time_us":11488,"prepare.run_wall_time_us":364630,"raft.queue_time_us":474630,"raft.run_cpu_time_us":32126,"raft.run_wall_time_us":158996,"replication_time_us":564231,"thread_start_us":1371600,"threads_started":4,"wal-append.queue_time_us":41148}]]}
I20260709 12:13:04.511020 18033 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.512369 17939 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.513123 17793 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:04.513641 17793 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:04.514720 17793 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.516253 17793 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:04.505267 18050 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:13:04.527999 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 140228us
W20260709 12:13:04.528373 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 64030us
I20260709 12:13:04.538225 10578 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:13:04.539499 10505 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:13:04.457329 17988 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.458150 17903 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.441845 18039 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.535598 18050 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:04.605275 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:04.608733 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.610317 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:04.619163 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:04.616920 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:04.622949 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.626047 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.628551 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:04.460429 18038 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.625419 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:04.629930 10580 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:04.634325 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:04.637691 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98419.1us
I20260709 12:13:04.551080 17906 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:04.639796 10585 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.650516 10582 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:04.629665 17903 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.649279 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.655550 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:04.660066 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 107704us
I20260709 12:13:04.633821 17988 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.637328 18033 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.646842 17789 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:04.666220 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.677977 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.693298 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.695438 10504 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
W20260709 12:13:04.699677 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50509.9us
I20260709 12:13:04.673184 17789 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.705466 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.703929 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:04.670164 17937 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:13:04.667631 11178 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.11.65:42153 timed out after 1.620s (SENT) [suppressed 3 similar messages]
W20260709 12:13:04.719273 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 63899.8us
I20260709 12:13:04.698746 18048 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.728837 10511 raft_consensus.cc:1275] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:04.738312 10585 raft_consensus.cc:2468] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:04.731966 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.687016 18047 consensus_queue.cc:1059] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.182s
I20260709 12:13:04.679293 18033 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.737725 10403 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:04.715021 17789 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:04.697943 17988 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.673627 17903 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:13:04.753859 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 86262.2us
I20260709 12:13:04.761330 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:04.767191 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 94909.3us
I20260709 12:13:04.770485 10430 raft_consensus.cc:2468] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:04.767978 10508 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
W20260709 12:13:04.776453 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 93087us
I20260709 12:13:04.774719 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:04.759332 18036 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:04.762491 18020 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.780887 18049 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:04.781394 18060 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:04.783252 18044 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:04.804515 18039 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.805666 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.783795 18035 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:04.784730 10437 raft_consensus.cc:1275] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:13:04.807051 17903 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.807412 18035 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:04.765087 18048 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.808171 17872 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.808480 18035 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.809682 18048 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.810545 10510 raft_consensus.cc:2393] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 15: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:04.811254 17988 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.811690 18034 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 16. Current term is 17.
I20260709 12:13:04.812204 18044 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.813160 18020 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.787379 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.814488 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:04.774261 18062 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:04.727252 18040 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.165s
I20260709 12:13:04.815840 18020 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.774712 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.806717 17883 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:13:04.818455 17946 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:04.818702 18049 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:04.818905 17946 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:04.819221 18049 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:04.819455 17946 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.820449 18049 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:04.685966 18050 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:04.821672 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 76757.6us
I20260709 12:13:04.822611 18050 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.810564 17872 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.795245 10580 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:04.798872 17985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945dd7b821a5493f8a9d6ca251795da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:04.797560 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:04.801425 11177 leader_election.cc:400] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19.
I20260709 12:13:04.804214 18060 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:04.814706 18044 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.795289 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:04.838270 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.779352 18038 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.848413 10434 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:04.848593 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.846807 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:04.851277 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.853711 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.858486 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.859110 10429 raft_consensus.cc:2468] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:13:04.863888 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.865705 10435 raft_consensus.cc:1275] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:04.869633 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.875483 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:04.877813 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55063.4us
W20260709 12:13:04.878163 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 72633.8us
I20260709 12:13:04.833870 18020 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:13:04.852296 17906 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.861382 10587 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:04.898847 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:04.861712 17948 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:04.902207 18060 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.903224 18033 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.796366 17937 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.876991 18025 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:13:04.914762 18059 consensus_queue.cc:1048] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
W20260709 12:13:04.930229 17822 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.005s
I20260709 12:13:04.930912 17822 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.933324 17822 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.934016 10513 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:04.934793 10587 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:04.934881 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46741.7us
I20260709 12:13:04.935477 18058 consensus_queue.cc:1048] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:04.907541 17879 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.943184 18043 consensus_queue.cc:1059] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:04.944173 18043 raft_consensus.cc:3055] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:04.944510 18043 raft_consensus.cc:740] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:04.945071 17789 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.945513 18043 consensus_queue.cc:260] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.946274 10511 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:04.947546 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:04.947878 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:04.942291 17946 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.952153 17789 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.894868 18040 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.950809 17946 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:04.954308 18040 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:04.958950 10583 raft_consensus.cc:1240] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:13:04.963526 10581 raft_consensus.cc:1240] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:04.963824 18040 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:04.938688 17879 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:04.946833 18043 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:04.969861 17941 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.747s	user 0.008s	sys 0.008s
I20260709 12:13:04.909013 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:04.972103 10479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.978526 18049 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.980082 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:04.980479 18040 consensus_queue.cc:1059] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:04.981285 18063 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:04.981453 18040 raft_consensus.cc:3055] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:04.981868 18040 raft_consensus.cc:740] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:04.905041 17909 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:04.982558 17909 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:04.982824 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26d3d17d58be40c8974c348ca146a951. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:04.987485 17909 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:04.982769 18040 consensus_queue.cc:260] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:04.996380 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87518.2us
I20260709 12:13:04.996701 18068 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:04.974135 17941 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:04.999069 10433 raft_consensus.cc:1240] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:04.999966 17941 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.001397 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.002465 18040 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:04.997303 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:05.009087 18049 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:13:05.009538 18068 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:05.011413 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:04.966606 10585 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:04.973443 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:05.019116 10478 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.027503 18068 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:05.030370 18051 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:13:05.033885 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.043987 10478 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:05.038592 18062 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:13:05.050066 18068 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:05.054108 18062 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:05.054639 18062 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:05.059422 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.060019 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.065501 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.075632 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:05.066548 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 93238.4us
I20260709 12:13:05.039147 18063 raft_consensus.cc:2804] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:05.092438 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:05.097079 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.102559 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.104080 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:05.072892 18037 raft_consensus.cc:3055] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:05.111846 10435 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:05.103503 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.104400 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.116328 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:05.116688 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.116873 17941 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.118512 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.116106 11214 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:05.095750 18068 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.140287 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.156358 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.114885 18037 raft_consensus.cc:740] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:05.110335 17946 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:13:05.164083 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.170116 10582 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:05.167708 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.167183 18049 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:05.173714 18049 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.176241 18049 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:05.181318 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:05.110816 18026 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:13:05.182709 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.183787 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.190779 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.131965 18063 raft_consensus.cc:697] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.206763 10585 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:05.210500 10504 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:13:05.211205 10504 pending_rounds.cc:85] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 12
W20260709 12:13:05.212967 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 50913.6us
I20260709 12:13:05.214278 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.218055 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.218760 10429 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:05.220693 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 117250us
I20260709 12:13:05.221658 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.223169 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.223734 17931 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.225340 18036 consensus_queue.cc:1059] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:05.226373 18036 raft_consensus.cc:3055] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:05.227811 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.167486 18060 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:05.211592 10504 pending_rounds.cc:107] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:13:05.235872 10504 raft_consensus.cc:2889] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.240883 10582 raft_consensus.cc:2468] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
I20260709 12:13:05.183626 18044 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:05.242316 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 80207.6us
I20260709 12:13:05.245677 18037 consensus_queue.cc:260] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.246961 18068 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.227967 17948 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.226779 18036 raft_consensus.cc:740] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:05.178058 17793 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
W20260709 12:13:05.257046 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.239013 17931 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:05.252328 18072 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:13:05.269608 10579 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:05.270318 18074 raft_consensus.cc:993] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:05.270825 18074 raft_consensus.cc:1081] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:05.271781 18036 consensus_queue.cc:260] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.273514 18036 raft_consensus.cc:3060] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:05.281464 10432 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:13:05.283020 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.284411 10479 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.285037 18044 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.286142 18050 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:05.227419 17822 consensus_queue.cc:1059] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:13:05.287432 18044 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:05.287701 17822 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:05.288061 18050 raft_consensus.cc:3055] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:05.288626 18050 raft_consensus.cc:740] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.251986 18061 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:05.289517 18050 consensus_queue.cc:260] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.290270 18042 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:05.251742 18060 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.290922 18042 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.284013 18063 consensus_queue.cc:237] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.292605 18060 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.299281 10429 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 23.
I20260709 12:13:05.313869 18062 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.314571 17909 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.316327 18082 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.316879 10433 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:05.316890 18082 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.319274 11212 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:05.290761 18029 consensus_queue.cc:1175] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 20 status { last_received { term: 20 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 20 index: 13 } }
I20260709 12:13:05.319764 18082 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.321537 17909 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.287464 17906 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:05.312172 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 53980us
I20260709 12:13:05.329978 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.325536 18035 raft_consensus.cc:993] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:13:05.333067 10581 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:05.348611 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.351209 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.352067 10435 raft_consensus.cc:2376] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21. Re-sending same reply.
I20260709 12:13:05.357452 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.361188 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.362658 17870 raft_consensus.cc:1081] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:13:05.364187 18035 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.364900 18035 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.364533 18046 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 11 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:05.373044 10433 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:13:05.373302 10513 raft_consensus.cc:1240] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:05.335553 18062 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.383728 10585 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:13:05.390911 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.391047 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.393635 10479 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:05.395694 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.399358 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.400128 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.399762 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:05.402537 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.406051 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.411037 18091 ts_tablet_manager.cc:933] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:05.394944 17906 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:13:05.415412 18089 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:05.415992 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.419404 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.424340 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.425767 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.427341 10585 raft_consensus.cc:1240] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:13:05.427811 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.429617 10431 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:13:05.430663 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.430769 17909 consensus_queue.cc:1048] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:05.432240 18060 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:05.436419 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:05.440024 10585 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:13:05.406828 17879 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:13:05.445816 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.446313 17879 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:05.446813 17879 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:05.402724 18030 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:05.450196 18030 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.451329 18030 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.400249 18035 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.454095 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:05.416008 18089 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.459369 17883 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:05.459614 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.405823 10432 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 11 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:05.459910 17883 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:05.434747 17985 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57904 (request call id 1168) took 2997 ms. Trace:
I20260709 12:13:05.477259 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.466691 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.509709 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:13:05.473914 18075 log.cc:927] Time spent T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.312s	user 0.003s	sys 0.000s
I20260709 12:13:05.520870 18091 tablet_copy_client.cc:323] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:13:05.522171 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:05.468650 17883 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.526737 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:05.527735 10506 raft_consensus.cc:2368] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:05.480846 11221 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 11 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:05.529186 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:05.531250 17948 consensus_queue.cc:1048] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:05.532126 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:05.539304 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:05.537268 10479 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:05.541630 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:05.543756 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:05.544559 10583 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:13:05.545676 17883 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.546744 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:05.547832 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.548626 18062 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:05.475790 17789 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:05.551498 17789 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.551855 18089 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:05.551905 18062 consensus_queue.cc:1059] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:05.553834 17789 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:05.477458 17985 rpcz_store.cc:276] 0709 12:13:02.437226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.437363 (+   137us) service_pool.cc:224] Handling call
0709 12:13:03.317487 (+880124us) ts_tablet_manager.cc:934] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:13:04.797300 (+1479813us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:05.322113 (+524813us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:05.322259 (+   146us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:05.433785 (+111526us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:05.434178 (+   393us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:05.434544 (+   366us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:05.434637 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":35177,"tablet-copy.queue_time_us":541236,"thread_start_us":538256,"threads_started":2}
I20260709 12:13:05.555886 17985 tablet_copy_client.cc:806] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:05.556720 17985 tablet_copy_client.cc:670] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
W20260709 12:13:05.558465 10552 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.558830 18060 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.560500 18042 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:05.572710 18037 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:05.514060 17988 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.580093 10478 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.580181 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:05.584898 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 47903.2us
I20260709 12:13:05.587426 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.593596 10553 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.599807 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.600944 10584 raft_consensus.cc:2368] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:05.591400 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:05.602790 17789 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:13:05.605832 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:05.601572 18039 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.609368 17978 consensus_queue.cc:1059] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:05.610373 17978 raft_consensus.cc:3055] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:13:05.610745 17978 raft_consensus.cc:740] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:05.611143 10583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:13:05.611996 10583 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:05.611673 17978 consensus_queue.cc:260] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.612469 10583 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:05.613387 17978 raft_consensus.cc:3060] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:05.615866 10581 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:13:05.618409 17807 raft_consensus.cc:2804] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:13:05.621906 17903 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:05.622400 17903 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:05.623525 17903 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.623965 18082 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:05.624691 17903 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:05.625557 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:05.600665 18056 raft_consensus.cc:3055] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:05.627264 17807 raft_consensus.cc:697] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.627419 18056 raft_consensus.cc:740] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:05.628257 18048 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:05.628639 17807 consensus_queue.cc:237] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.629637 17789 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.630174 18019 consensus_queue.cc:1059] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:05.632685 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.633685 18025 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:05.599733 18050 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:05.656477 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60263.2us
I20260709 12:13:05.715878 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:05.659588 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68674.5us
I20260709 12:13:05.599665 10505 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41074
I20260709 12:13:05.603401 10478 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.717303 10505 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:05.717875 10505 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:05.726701 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.729468 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 96890.3us
W20260709 12:13:05.731066 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 127744us
W20260709 12:13:05.705199 17874 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:13:05.731802 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:05.710022 18079 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.536s	user 0.000s	sys 0.003s
W20260709 12:13:05.736287 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 186883us
I20260709 12:13:05.738528 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.730095 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:05.732801 10478 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.742856 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.693804 18042 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:05.751313 10403 leader_election.cc:400] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:05.751667 10403 leader_election.cc:403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:13:05.752362 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.765172 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.769497 10581 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:13:05.769630 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:05.741707 18029 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:05.770375 10429 raft_consensus.cc:2376] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15. Re-sending same reply.
I20260709 12:13:05.770882 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.776175 10430 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:13:05.771605 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.744751 18068 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:05.782486 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.745401 18039 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:05.745025 18074 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:05.749541 17988 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.787017 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.756405 18020 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:05.759865 18038 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:05.726045 18025 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.771276 18044 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.800834 10581 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:13:05.740896 18035 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:13:05.803568 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.805016 10433 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
I20260709 12:13:05.804291 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:05.800302 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:05.812686 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 69332.4us
I20260709 12:13:05.821663 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:05.783802 17937 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:05.826956 10505 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:05.781881 18062 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:05.800403 18030 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:05.862633 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59165us
I20260709 12:13:05.863884 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.848425 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:05.851416 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44860.6us
I20260709 12:13:05.871444 10437 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:05.873741 10431 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:05.803719 18068 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:05.880023 10513 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:13:05.886793 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.865002 17793 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:05.804147 18074 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:05.811327 18059 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:13:05.912935 10553 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.884683 18026 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:05.918883 10512 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:13:05.921446 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.805020 18056 consensus_queue.cc:260] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.897010 18047 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:05.931704 10511 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:05.923868 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.893822 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 26d3d17d58be40c8974c348ca146a951 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:13:05.923084 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:05.880573 18089 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:13:05.907229 18055 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:13:05.941432 18100 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:13:05.943216 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 26d3d17d58be40c8974c348ca146a951 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-26d3d17d58be40c8974c348ca146a951
I20260709 12:13:05.948076 10508 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:05.915477 17939 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:05.871025 17789 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:05.953554 18096 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:05.921931 18034 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.928084 18038 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:05.940107 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:05.973384 10583 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:05.938902 17988 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:05.982542 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 249785us
W20260709 12:13:05.964824 18046 consensus_meta.cc:287] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.002s
W20260709 12:13:05.935449 18036 consensus_meta.cc:287] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.661s	user 0.017s	sys 0.000s
W20260709 12:13:06.007373 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 185300us
W20260709 12:13:06.016125 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 76150.8us
I20260709 12:13:06.017417 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:06.024068 10403 leader_election.cc:400] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 14. Current term is 15.
I20260709 12:13:06.025615 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:06.028698 18061 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:06.005741 17778 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:06.034257 11178 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:06.035584 18043 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:06.036096 18043 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:06.036487 18043 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:06.038434 17792 raft_consensus.cc:3055] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:06.038911 17792 raft_consensus.cc:740] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:06.039883 17792 consensus_queue.cc:260] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.040993 17792 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:06.043726 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:05.959901 18068 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:13:06.164433 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 147147us
I20260709 12:13:05.939754 18047 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.044675 18106 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:06.006657 18056 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:06.203610 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 159970us
W20260709 12:13:06.208133 11178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:06.211649 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 142204us
I20260709 12:13:06.243875 10580 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:13:06.162022 17978 consensus_meta.cc:287] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.003s
I20260709 12:13:06.140385 17939 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:06.255419 10586 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:06.245579 10580 raft_consensus.cc:2955] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 11 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:06.263259 10579 raft_consensus.cc:1240] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:06.254406 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 55949.9us
I20260709 12:13:06.136962 17883 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 12:13:06.248178 18019 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.290195 18019 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.291780 18074 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:13:06.293094 18019 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:06.266237 18104 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.296066 18104 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:06.276077 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 70208.3us
I20260709 12:13:06.194644 18049 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:06.281643 10553 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:06.257952 17872 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.212806 18030 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:13:06.258797 18036 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.305868 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:06.343654 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:13:06.314074 17874 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:13:06.365942 10437 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:06.310158 18025 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:06.314680 18038 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:06.309116 18104 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:06.343415 17988 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:06.381575 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 172726us
I20260709 12:13:06.346014 18061 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:13:06.319686 17872 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.394454 11178 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:06.354611 18026 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:13:06.358333 18036 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.365535 18044 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:13:06.416004 18110 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:06.416805 18110 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:06.419831 18099 log.cc:927] Time spent T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:13:06.422209 10437 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:06.425563 18036 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:06.431346 18050 consensus_meta.cc:287] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.691s	user 0.000s	sys 0.015s
I20260709 12:13:06.409801 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:06.413628 10523 tablet_copy_source_session.cc:215] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:13:06.398615 17879 consensus_meta.cc:287] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.951s	user 0.008s	sys 0.007s
I20260709 12:13:06.408588 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:06.436873 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.439168 10431 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:13:06.436905 10553 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:06.406754 17978 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.469399 11178 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:06.477742 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.480265 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.481465 10580 raft_consensus.cc:2393] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:06.465479 18020 raft_consensus.cc:2804] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:06.478796 10579 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:06.460793 17879 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.497848 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:06.488407 18047 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:06.469321 17978 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.502380 10504 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:06.506356 18120 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:06.506925 18120 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:06.507306 18120 raft_consensus.cc:3060] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:13:06.510957 18086 log.cc:927] Time spent T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:13:06.511152 18118 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:06.511698 18118 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:06.511835 18129 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.512135 18118 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:06.512465 18129 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.511698 18109 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:13:06.515793 18074 raft_consensus.cc:493] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:06.516436 18074 raft_consensus.cc:515] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.516961 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.517638 10586 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:06.491247 18123 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
W20260709 12:13:06.521217 18131 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:13:06.525468 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:06.503415 17822 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:06.530735 17822 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.531208 17822 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:06.533074 18129 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:06.534530 18129 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:06.518813 18074 leader_election.cc:290] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:06.522845 18110 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:06.540328 18115 log.cc:927] Time spent T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:13:06.540925 17874 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=2 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:06.534906 18129 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:13:06.541570 17903 consensus_meta.cc:287] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.916s	user 0.018s	sys 0.000s
I20260709 12:13:06.541743 10505 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:06.548070 18043 consensus_meta.cc:287] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.002s
I20260709 12:13:06.548314 10505 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41074 (request call id 3639) took 1009 ms. Trace:
I20260709 12:13:06.548595 18043 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.548723 10505 rpcz_store.cc:276] 0709 12:13:05.585151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.599496 (+ 14345us) service_pool.cc:224] Handling call
0709 12:13:06.541070 (+941574us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b4c00044240. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c90daba20 00007f2c90dbca29 00007f2c90dbd13f 00007f2c90d812ab 00007f2c90daef3c 00007f2c90daf3fe 00007f2c9103c88b 00007f2c92a3b495 00007f2c92a8c30f 00007f2c92ad6a55 00007f2c97aa6cd0
0709 12:13:06.548207 (+  7137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43876096}
W20260709 12:13:06.542284 18037 consensus_meta.cc:287] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.969s	user 0.017s	sys 0.000s
I20260709 12:13:06.549846 11217 raft_consensus.cc:1240] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:06.544806 17872 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:06.497799 18020 raft_consensus.cc:697] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:06.551709 17793 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:06.504262 18046 consensus_queue.cc:1059] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:13:06.552278 17793 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:06.552971 18046 raft_consensus.cc:3055] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:06.553355 18046 raft_consensus.cc:740] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:06.554234 18046 consensus_queue.cc:260] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.551432 18020 consensus_queue.cc:237] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.555847 17978 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:06.557710 17879 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:06.558604 18129 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:06.586856 10583 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:06.569873 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:06.573336 18120 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.574097 10505 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:13:06.542224 17874 tablet_bootstrap.cc:492] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:13:06.563006 11214 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:06.572582 17906 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:06.564261 10513 raft_consensus.cc:1240] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:06.596948 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:06.598140 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:06.593185 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:06.587941 10511 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:06.599715 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:06.613165 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 112671us
I20260709 12:13:06.616091 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:06.616853 10436 raft_consensus.cc:3055] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:06.620440 11215 raft_consensus.cc:1275] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:13:06.620956 11215 pending_rounds.cc:85] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting all ops after (but not including) 11
I20260709 12:13:06.621246 11215 pending_rounds.cc:107] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:13:06.621541 11215 raft_consensus.cc:2889] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:06.623939 11218 raft_consensus.cc:1240] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:06.625372 10479 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:06.626408 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:06.625643 10508 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 22: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:06.617216 10436 raft_consensus.cc:740] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:06.631769 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:06.632400 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 107020us
I20260709 12:13:06.640884 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:06.643159 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:06.644467 10438 raft_consensus.cc:1275] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:13:06.644897 10438 pending_rounds.cc:85] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 13
I20260709 12:13:06.645161 10438 pending_rounds.cc:107] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:13:06.645401 10438 raft_consensus.cc:2889] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:06.649870 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.650579 10582 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:13:06.652011 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:06.653370 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.654232 10586 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:13:06.660411 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:06.661480 10583 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 17: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:06.661712 10433 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:06.662129 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:06.601605 18043 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:06.669072 10434 raft_consensus.cc:2410] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:13:06.618703 10437 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:06.660952 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:06.675669 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:06.676357 10513 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:06.666072 10436 consensus_queue.cc:260] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.678639 10436 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:06.645370 17807 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:06.683077 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc98cd5edc9240fba53978ed59191cbd" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:06.690276 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:06.692298 10505 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:13:06.679648 17874 ts_tablet_manager.cc:1403] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 3.053s	user 0.054s	sys 0.033s
I20260709 12:13:06.709712 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:06.644920 17879 raft_consensus.cc:493] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:06.677045 18068 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:13:06.733016 18122 log.cc:927] Time spent T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:13:06.733791 18125 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:06.733995 18126 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 12:13:06.737147 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.739440 18138 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:13:06.690150 18058 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:06.743515 18058 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.747735 10583 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:13:06.753546 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:06.703387 17978 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:06.755738 10436 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:06.758065 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:06.758630 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:06.759157 11215 raft_consensus.cc:2410] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:13:06.759313 11219 raft_consensus.cc:3060] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:06.759645 18133 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:06.760094 18133 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.760449 18133 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:06.732343 17879 raft_consensus.cc:515] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.711995 18024 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.769371 18024 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.757499 17978 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.773869 10579 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:06.772022 18044 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:06.783529 17792 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.737s	user 0.013s	sys 0.000s
I20260709 12:13:06.787087 18120 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:06.780418 17978 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:06.798461 18024 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:06.800614 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:06.772653 18063 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:13:06.782166 18044 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.837513 18062 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:06.849452 18062 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:06.838151 18106 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:06.804136 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:06.853322 10504 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41074 (request call id 3634) took 1400 ms. Trace:
I20260709 12:13:06.853845 10504 rpcz_store.cc:276] 0709 12:13:05.453272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.453769 (+   497us) service_pool.cc:224] Handling call
0709 12:13:06.853077 (+1399308us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54002c5bd4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:06.853238 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292563840}
W20260709 12:13:06.904943 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 85379.2us
I20260709 12:13:06.913563 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:13:06.916271 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 55914.2us
I20260709 12:13:06.850760 18062 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:06.920060 18069 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:06.920312 18062 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:06.920681 17874 raft_consensus.cc:359] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:06.921696 17874 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:06.922065 18138 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:06.925499 17985 tablet_copy_client.cc:538] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:06.925806 18118 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.928771 18138 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:06.920876 18069 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.931381 11218 raft_consensus.cc:1240] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:06.755410 18068 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:06.937790 17807 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:06.846210 10336 catalog_manager.cc:5697] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 11 to 12, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 12 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:06.946720 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:06.947954 10586 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:13:06.936075 11219 raft_consensus.cc:2468] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:06.952221 17792 consensus_queue.cc:1048] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:13:06.945235 17909 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.979869 18058 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:06.982098 10508 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 15. Current term is 16. Ops: []
I20260709 12:13:06.988606 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:06.989497 10433 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:07.013725 10479 leader_election.cc:304] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:07.043387 11177 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:07.076845 10438 raft_consensus.cc:2468] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
W20260709 12:13:07.128561 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 85295.2us
W20260709 12:13:07.179251 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 136634us
I20260709 12:13:06.949555 18042 consensus_queue.cc:1059] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
W20260709 12:13:07.052783 17822 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.521s	user 0.014s	sys 0.009s
I20260709 12:13:06.922499 17874 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:07.083966 18123 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:07.040391 18054 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:13:07.130123 18060 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:06.981696 17909 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:07.131520 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.164157 10434 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:13:07.169420 18069 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:07.231096 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47730.8us
I20260709 12:13:07.231976 10479 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:07.152163 18058 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.233031 18058 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:07.187197 17879 leader_election.cc:290] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.139715 18113 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.235877 18113 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.236907 18118 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.151337 18073 raft_consensus.cc:2749] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:07.142735 18091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26d3d17d58be40c8974c348ca146a951. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:13:07.149013 18129 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.562s	user 0.016s	sys 0.004s
I20260709 12:13:07.135847 18112 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.151675 17978 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.191551 18046 raft_consensus.cc:3060] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:07.193979 18068 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.140830 18125 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:07.143692 18133 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.197196 10328 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 172) took 5565 ms. Trace:
I20260709 12:13:07.278512 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:07.199301 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55466 (request call id 3598) took 1418 ms. Trace:
I20260709 12:13:07.199419 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:07.279661 10578 rpcz_store.cc:276] 0709 12:13:05.781199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.781427 (+   228us) service_pool.cc:224] Handling call
0709 12:13:05.798481 (+ 17054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.816372 (+ 17891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.817112 (+   740us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.817492 (+   380us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:05.817657 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.198941 (+1381284us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b54001d60d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:13:07.199020 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.199175 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38420224}
I20260709 12:13:07.200775 10336 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 21 to 22, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:07.181167 17793 consensus_meta.cc:287] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.628s	user 0.014s	sys 0.018s
I20260709 12:13:07.010001 18024 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:07.140553 18044 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:07.239275 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:07.242074 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.215265 17822 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.218550 18109 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.277762 10436 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:07.201320 18042 raft_consensus.cc:3055] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:07.284986 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:07.285812 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.269155 17872 raft_consensus.cc:2804] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:07.279109 10328 rpcz_store.cc:276] 0709 12:13:01.631383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.634049 (+  2666us) service_pool.cc:224] Handling call
0709 12:13:07.197088 (+5563039us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:04.771963 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:04.787373 (+ 15410us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:04.787430 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:13:04.787450 (+    20us) tablet.cc:662] Decoding operations
0709 12:13:04.787897 (+   447us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:04.787977 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:04.788012 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:04.788320 (+   308us) tablet.cc:701] Row locks acquired
0709 12:13:04.788358 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:13:04.788550 (+   192us) write_op.cc:270] Start()
0709 12:13:04.788755 (+   205us) write_op.cc:276] Timestamp: P: 1783599184788482 usec, L: 0
0709 12:13:04.788777 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:13:04.813552 (+ 24775us) log.cc:844] Serialized 530 byte log entry
0709 12:13:05.428713 (+615161us) op_driver.cc:464] REPLICATION: finished
0709 12:13:05.817448 (+388735us) write_op.cc:301] APPLY: starting
0709 12:13:05.841492 (+ 24044us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:06.300583 (+459091us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:06.340094 (+ 39511us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:06.341413 (+  1319us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:06.341715 (+   302us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:13:06.341787 (+    72us) write_op.cc:312] APPLY: finished
0709 12:13:06.342082 (+   295us) log.cc:844] Serialized 25 byte log entry
0709 12:13:06.342343 (+   261us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:06.342482 (+   139us) write_op.cc:454] Released schema lock
0709 12:13:06.342548 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:06.369548 (+ 27000us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:06.377229 (+  7681us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1053748,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":355343,"apply.run_cpu_time_us":6674,"apply.run_wall_time_us":544527,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":238536,"prepare.run_cpu_time_us":96451,"prepare.run_wall_time_us":351013,"raft.queue_time_us":102861,"raft.run_cpu_time_us":65961,"raft.run_wall_time_us":281442,"replication_time_us":639768,"thread_start_us":772318,"threads_started":4,"wal-append.queue_time_us":306323}]]}
I20260709 12:13:07.289327 17909 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.289896 18073 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.290063 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.290506 17807 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.310261 10505 raft_consensus.cc:2393] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:07.290910 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.291956 17789 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.294696 10479 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:07.265879 18129 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.230504 18123 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.303794 10437 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:07.312985 10479 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:07.285825 17793 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.297888 18042 raft_consensus.cc:740] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:07.314230 10403 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:07.322132 18133 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.270371 18112 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.310674 18058 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:07.295487 18125 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.311679 17872 raft_consensus.cc:697] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:07.301507 18109 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.320235 18044 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.319890 17941 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:07.323086 18048 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:07.332021 18074 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:07.321094 17789 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.315145 18073 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.332106 17793 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.334744 18036 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:07.323268 18019 consensus_queue.cc:1048] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:13:07.339843 18137 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.318128 17807 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.341367 18062 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:07.395848 18062 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.397209 17807 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.362514 18137 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.369022 17985 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:13:07.372463 17874 ts_tablet_manager.cc:1434] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.641s	user 0.010s	sys 0.010s
I20260709 12:13:07.360301 18048 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:07.380721 18073 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.353236 17941 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.382748 18042 consensus_queue.cc:260] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.383541 17872 consensus_queue.cc:237] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.343053 18089 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.385150 17789 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.385440 10509 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:07.386090 17822 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.386618 18123 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.387187 18036 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:07.387996 18129 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.357299 18133 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.391821 18026 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.398694 17978 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.340691 18105 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:13:07.415187 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:07.399916 18062 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.445230 10438 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
W20260709 12:13:07.408030 17874 ts_tablet_manager.cc:1467] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Tablet startup took 691ms
I20260709 12:13:07.448045 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.452697 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.351733 18044 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.451893 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.455610 10429 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48986 (request call id 3364) took 1019 ms. Trace:
I20260709 12:13:07.462755 10429 rpcz_store.cc:276] 0709 12:13:06.435926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.436062 (+   136us) service_pool.cc:224] Handling call
0709 12:13:06.436367 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:06.438317 (+  1950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:06.438363 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:06.451304 (+ 12941us) log.cc:844] Serialized 26 byte log entry
0709 12:13:07.440769 (+989465us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b180018ec80. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c926874bc 00007f2c8f9d548b 00007f2c9882f9a9 00007f2c8f9f169b 00007f2c8f9ef93f 00007f2c8f9f1c02 00007f2c9267920b 00007f2c9267ed23 00007f2c9267f0d1
0709 12:13:07.441117 (+   348us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:07.441175 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.441290 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:07.441378 (+    88us) raft_consensus.cc:1685] finished
0709 12:13:07.441395 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:07.455524 (+ 14129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7975,"raft.queue_time_us":1645,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":15790,"spinlock_wait_cycles":35064576,"thread_start_us":129267,"threads_started":1,"wal-append.queue_time_us":144493}
I20260709 12:13:07.399499 17906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:38164 (request call id 17) took 3022 ms. Trace:
I20260709 12:13:07.484670 18069 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
W20260709 12:13:07.534433 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66994.8us
I20260709 12:13:07.505446 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:07.535425 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 206557us
I20260709 12:13:07.449946 18111 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:07.541213 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:07.541874 10586 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:07.490550 18138 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.525035 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:07.526173 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:07.530869 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:07.532059 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:07.517452 11178 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:13:07.536573 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 84001.2us
I20260709 12:13:07.417836 17870 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:07.534425 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.539249 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:07.446786 18034 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:07.550446 10511 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:07.554353 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.461604 18042 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:07.562517 10513 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:07.558002 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:07.570005 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38ead18e1a814c8b958f80396faa3170" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.572605 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.564006 17903 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.575807 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.446822 18026 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.446780 18133 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.584057 10582 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:13:07.585238 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:07.427963 18089 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:07.592865 10582 pending_rounds.cc:85] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Aborting all ops after (but not including) 13
I20260709 12:13:07.599602 10582 pending_rounds.cc:107] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:13:07.600034 10582 raft_consensus.cc:2889] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:07.550599 18055 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.600215 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:07.605333 10506 raft_consensus.cc:2393] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 16: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:07.566517 18098 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.532737 18038 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:13:07.603394 10429 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:07.601807 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 84514us
I20260709 12:13:07.607589 10509 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:13:07.609212  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:07.618712 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.619922 11215 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:13:07.621327 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:07.623409 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.608081 10479 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:07.613453 10438 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:13:07.615367 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.636802 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:07.638178 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.613024 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.585568 18050 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.548142 18069 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.648711 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.661849 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.662731 10584 raft_consensus.cc:2410] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:13:07.649821 10433 raft_consensus.cc:1275] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:13:07.577464 17883 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:13:07.666169 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.606019 17778 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:07.675868 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:07.675686 10581 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:07.605793 18060 raft_consensus.cc:2804] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:13:07.694202 18069 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:07.659727 18079 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:13:07.652524 18036 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.695694 10586 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:13:07.556454 18048 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.700484 18026 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.579041 17870 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:07.703500 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98611.2us
I20260709 12:13:07.567073 18034 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:07.661532 17937 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:07.719848 11215 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Advancing to term 20
I20260709 12:13:07.724577 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:07.716696 18147 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:13:07.748440 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.704619 18112 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:07.748425 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 163271us
I20260709 12:13:07.750946 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.711704 18155 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:07.647014 18098 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:07.725539 18149 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:13:07.726724 17937 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.712218 18043 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.723266 18060 raft_consensus.cc:697] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:07.763764 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:07.636241 18055 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:07.665875 18050 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.741263 18074 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.769802 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.756822 18137 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.772011 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:07.728999 18063 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.746039 18125 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.780994 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.802533 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.773762 18123 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.787161 18160 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.788901 17870 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.789744 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:07.795938 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80de1e7bce9c4d34b0583315cafbc371" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:07.760627 18104 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.767552 10429 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:07.770947 10478 leader_election.cc:304] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:07.777077 18112 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.774300 17937 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:07.797438 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:07.757143 18109 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.805189 10579 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:07.794242 18020 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:07.532966 17906 rpcz_store.cc:276] 0709 12:13:04.377534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.383691 (+  6157us) service_pool.cc:224] Handling call
0709 12:13:04.507030 (+123339us) log.cc:844] Serialized 492 byte log entry
0709 12:13:04.821278 (+314248us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b180030eb60. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f96cc0a 00007f2c990ac219 00007f2c8f9f4a40 00007f2c8f9ef4e4 00007f2c8f9ef455 00007f2c92a38b0b 00007f2c92abdb63 00007f2c92ae5752 00007f2c97aa5d29
0709 12:13:05.035376 (+214098us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b180030eb60. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:13:05.260184 (+224808us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b5400040220. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92a58142 00007f2c92a38eaa 00007f2c92a3fd53 00007f2c92a3fcd9 00007f2c92a3fc69 00007f2c92a3fc31 00007f2c92a3edfd 0000000000388634 0000000000388479 00007f2c8f9f2475
0709 12:13:05.388895 (+128711us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b180030eb60. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:13:07.366667 (+1977772us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47934,"raft.queue_time_us":98304,"raft.run_cpu_time_us":6684,"raft.run_wall_time_us":642868,"spinlock_wait_cycles":33043456,"thread_start_us":222816,"threads_started":1,"wal-append.queue_time_us":318885,"wal-append.run_cpu_time_us":2167,"wal-append.run_wall_time_us":1103655}
I20260709 12:13:07.808944 18063 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.809438 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:07.799351 17883 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:07.779045 18043 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.841182 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
W20260709 12:13:07.844197 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 73345.9us
I20260709 12:13:07.845713 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.854976 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:07.858914 11215 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:07.808558 18123 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.807478 18074 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.840097 18044 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:07.860510 18048 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:07.810822 18104 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.841693 17883 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:07.850857 18105 raft_consensus.cc:2749] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:07.846282 18060 consensus_queue.cc:237] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.957293 17903 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:07.808691 18160 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.939714 18098 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.940219 18043 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:07.831606 18033 consensus_queue.cc:1059] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.033s
I20260709 12:13:07.861776 18133 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:07.831897 18112 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:07.962158 18033 raft_consensus.cc:3055] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:07.860450 18106 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:07.962818 18033 raft_consensus.cc:740] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:07.950592 18034 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:07.967953 10506 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 22: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:07.943027 17883 raft_consensus.cc:3060] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:07.951589 18161 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:13:07.839872 18020 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:07.955667 18168 log.cc:927] Time spent T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:13:07.454720 17874 ts_tablet_manager.cc:1469] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Trace:
0709 12:12:59.929832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.930017 (+   185us) service_pool.cc:224] Handling call
0709 12:13:00.422234 (+492217us) ts_tablet_manager.cc:934] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
0709 12:13:01.141531 (+719297us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:01.729606 (+588075us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:01.729890 (+   284us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:02.064108 (+334218us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:02.064739 (+   631us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:02.065303 (+   564us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:02.065403 (+   100us) inbound_call.cc:177] Queueing success response
0709 12:13:03.615035 (+1549632us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:13:06.731543 (+3116508us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:13:06.807112 (+ 75569us) tablet_replica.cc:261] Starting instrumentation
0709 12:13:06.808020 (+   908us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":1652,"mutex_wait_us":16858,"raft.queue_time_us":12435,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":9103,"tablet-copy.queue_time_us":327337,"tablet-download-pool.queue_time_us":20556,"tablet-download-pool.run_cpu_time_us":2051,"tablet-download-pool.run_wall_time_us":392788,"thread_start_us":271809,"threads_started":2}
I20260709 12:13:07.962240 18111 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:07.949977 18074 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.963270 18106 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:07.970340 18113 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.869668 18164 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
I20260709 12:13:07.972553 18112 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:07.973201 18063 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:07.976984 18152 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.510s	user 0.001s	sys 0.001s
I20260709 12:13:07.968984 18133 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.997923 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:07.998785 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:08.000079 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:08.012192 18055 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.013218 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.007647 18111 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:08.004858 18034 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.014484 18089 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.039541 18160 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.027189 18133 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.041492 18123 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.045483 17937 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:13:08.052386 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 110526us
I20260709 12:13:08.054388 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:08.051915 18046 consensus_meta.cc:287] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.779s	user 0.018s	sys 0.000s
I20260709 12:13:08.064637 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.057689 18164 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.070344 18133 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.072377 18104 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.073465 18106 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.074218 18020 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.075182 17941 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.035710 18034 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.078821 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:08.055979 18050 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.082821 10504 raft_consensus.cc:2468] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:08.087901 11217 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:08.091480 10478 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.097719 18035 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.098337 18035 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.065135 18160 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:08.107405 18050 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:08.119927 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.087280 18130 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:08.119210 17941 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.087534 18020 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.124142 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.129678 10583 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 20: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:08.130707 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.132264 10586 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 22.
I20260709 12:13:08.132582 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.134160 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.127943 17941 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:08.131211 18020 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.136402 17941 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.133335 10583 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:08.139911 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.140952 18166 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.141535 18166 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:08.142226 18034 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.143987 18166 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.136960 18020 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.140463 17937 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.145723 11178 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.147099 18035 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.147197 18106 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:08.147554 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.148929 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.149326 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.150112 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.150566 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.150725 10512 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 19.
I20260709 12:13:08.156450 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.156575 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.157029 10587 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:08.161321 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.163331 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.168913 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.169230 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.170171 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.171447 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.173938 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.162799 18038 consensus_queue.cc:1059] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:13:08.170243 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.177469 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.184051 10582 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:08.180750 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.181713 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.164021 10584 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:13:08.190057 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.191040 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.192341 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.193799 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.174964 18091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57904 (request call id 1199) took 2928 ms. Trace:
I20260709 12:13:08.185700 10511 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:08.196388 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.197114 10586 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 13.
I20260709 12:13:08.198323 10585 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 20: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:08.209107 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.215790 18111 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.217768 10505 raft_consensus.cc:1275] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:13:08.220053 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.209390 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.220780 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.221201 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:08.222078 10510 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:08.221684 10513 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:08.225682 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:08.227170 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:08.229254 18180 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:13:08.230548 18111 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.232939 18133 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.236409 18033 consensus_queue.cc:260] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.238301 18033 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:08.226284 10509 raft_consensus.cc:2376] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19. Re-sending same reply.
I20260709 12:13:08.194449 18091 rpcz_store.cc:276] 0709 12:13:05.246676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.250699 (+  4023us) service_pool.cc:224] Handling call
0709 12:13:05.411412 (+160713us) ts_tablet_manager.cc:934] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:13:06.980891 (+1569479us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:07.959331 (+978440us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:07.959480 (+   149us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:08.174091 (+214611us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:08.174468 (+   377us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:08.174820 (+   352us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:08.174888 (+    68us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":32587,"tablet-copy.queue_time_us":139625,"thread_start_us":201674,"threads_started":2}
I20260709 12:13:08.240257 18091 tablet_copy_client.cc:806] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:08.240927 18091 tablet_copy_client.cc:670] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:08.249686 10478 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.249475 17883 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.250679 17906 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.251284 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:08.251957 10429 raft_consensus.cc:606] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.252296 10429 raft_consensus.cc:614] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:08.253542 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:08.228582 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:08.223465 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:08.270745 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47465.7us
I20260709 12:13:08.272449 10437 raft_consensus.cc:1240] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:08.272856 18166 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:08.273455 18166 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.274051 18166 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:08.275451 10510 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:13:08.277045 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:08.277794 11221 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:08.279914 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.279867 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.280759 10429 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:08.269955 10403 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:08.282387 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.266950 10479 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.289515 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.292173 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.290189 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:08.302709 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.303201 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.279549 18035 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:08.299690 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.303227 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.269300 17883 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.267267 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:08.292886 18133 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:08.313393 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.303565 18068 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.324795 11217 raft_consensus.cc:2468] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:08.302160 18089 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:08.309700 18058 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:08.309118 18043 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:08.335790 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 68608.3us
I20260709 12:13:08.335906 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:08.327167 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:08.311123 18125 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:08.332944 10552 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:08.348794 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.349695 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.347541 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:08.328258 18036 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:13:08.388921 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56116.4us
I20260709 12:13:08.352299 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:08.310189 18048 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:13:08.436875 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:08.343706 18019 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:08.396226 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.398137 18160 consensus_queue.cc:1059] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:13:08.398692 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.381145 17906 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:08.397420 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-c545de01b7fb4c89946871c1d522006f received from {username='slave'} at 127.0.0.1:36746
I20260709 12:13:08.410812 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.411911 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.412830 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.460000 10436 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:08.420898 18055 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:08.421443 18020 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:08.426369 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.431046 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.382133 17941 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:08.327091 18098 consensus_queue.cc:1059] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:13:08.399865 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:08.439254 18030 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:08.386413 18111 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:08.326174 18162 consensus_queue.cc:1048] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 12:13:08.391894 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:08.440062 18112 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:13:08.444057 18019 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.314831 18035 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:08.349295 18165 consensus_queue.cc:1048] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:13:08.341107 18058 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.485822 10431 raft_consensus.cc:1240] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:08.485886 10505 raft_consensus.cc:2393] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:08.486524 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.488052 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.350345 18106 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.486063 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:08.510234 10433 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:08.461799 17906 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.476842 18030 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:08.329725 18054 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.139s
I20260709 12:13:08.474152 18111 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.512920 10430 raft_consensus.cc:2393] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:08.519351 10553 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.486042 18112 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:08.498294 18019 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:08.494279 18055 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:08.472605 17941 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.470947 18020 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.511370 18058 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:08.514457 17906 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:08.520109 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-c545de01b7fb4c89946871c1d522006f on tablet c545de01b7fb4c89946871c1d522006f with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.520414 18111 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:08.518411 18030 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:08.527930 18112 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:08.548298 10433 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:13:08.521178 18046 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:13:08.532939 18038 raft_consensus.cc:2804] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:08.533591 18133 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:08.558111 10479 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:08.523267 18106 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.549256 17883 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:08.589340 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.545939 18068 consensus_queue.cc:1059] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:13:08.602663 10586 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:13:08.607340 10479 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.584192 18038 raft_consensus.cc:697] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:08.605252 10586 raft_consensus.cc:606] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:08.574312 18024 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:13:08.599882 17807 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:08.610285 17872 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:08.583518 18035 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.623474 17822 raft_consensus.cc:3055] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:13:08.656787 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 138181us
I20260709 12:13:08.625890 17879 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.625286 18196 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:08.628449 17807 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:08.664477 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 72852.4us
I20260709 12:13:08.664011 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:08.662014 18183 log.cc:927] Time spent T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:13:08.596942 17883 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.678463 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.661371 17822 raft_consensus.cc:740] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:08.615020 18055 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:08.668558 17807 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:08.686761 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.668684 18196 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.640843 18133 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:08.692193 18181 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:13:08.675920 17883 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.694509 18194 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.690201 18193 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:13:08.695737 18190 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:08.694062 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.680115 17879 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.698524 17941 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.715090 17903 raft_consensus.cc:3055] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:13:08.701536 18185 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:13:08.723775 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.702327 18189 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:08.730772 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:08.734076 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 294367us
I20260709 12:13:08.722038 18020 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.743985 10505 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:13:08.724517 17903 raft_consensus.cc:740] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:08.682693 18038 consensus_queue.cc:237] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.749253 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.742635 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.724771 18113 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:08.737916 18194 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.753226 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.736660 18106 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.729393 17879 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.770830 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.775048 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.726130 17822 consensus_queue.cc:260] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.777761 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.775511 18091 tablet_copy_client.cc:538] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:13:08.776904 18033 consensus_meta.cc:287] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.538s	user 0.014s	sys 0.005s
I20260709 12:13:08.756896 18191 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:08.771389 18089 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:08.809083 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.817441 18205 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:08.776674 17870 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:08.842429 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 87287us
I20260709 12:13:08.812000 17941 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:08.836926 17822 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:08.821568 18201 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.863075 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.811632 17903 consensus_queue.cc:260] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.855626 18194 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.871227 17903 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:08.871943 18073 raft_consensus.cc:3055] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:13:08.878429 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:08.887213 18200 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:13:08.890635 18030 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.891475 17879 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.900951 18030 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.900692 18058 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.880230 18073 raft_consensus.cc:740] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:08.929941 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.864980 18201 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.931946 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.932638 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.935245 17778 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:08.935113 18111 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.936259 18058 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:08.909372 17807 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.937392 18196 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.937548 18111 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.938437 10436 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:08.939067 17807 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:08.939898 18201 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:08.940621 18166 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.009s
I20260709 12:13:08.940382 17906 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.941084 18166 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:08.942389 10510 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 20: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:08.942691 17906 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.943321 10582 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:13:08.945993 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.955586 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.956166 10587 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
I20260709 12:13:08.957695 11178 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.958901 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.950848 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:08.960242 11177 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:08.968294 10553 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.970029 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.975070 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.976140 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:08.980535 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:08.904093 18073 consensus_queue.cc:260] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:08.980643 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.989926 10434 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:08.953395 18165 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.992836 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:08.993505 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.993927 10430 raft_consensus.cc:1240] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:08.992434 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:08.965631 18060 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.969547 18197 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:08.994313 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:08.969813 18195 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:08.970376 18123 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:08.988654 18020 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:08.989076 18033 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:08.995378 18138 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:09.004366 18073 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:08.990859 18166 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:08.992120 18038 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:08.991612 18106 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:09.046164 18106 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.046628 18106 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.035199 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.003145 18198 consensus_queue.cc:1059] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:13:09.051940 18198 raft_consensus.cc:3055] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:09.052320 18198 raft_consensus.cc:740] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:09.052445 18204 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:09.039227 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:09.053615 11217 raft_consensus.cc:2393] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:09.053318 18198 consensus_queue.cc:260] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.054612 18198 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:09.008500 17883 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.040295 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:09.004269 18191 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:09.043459 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:09.062821 10507 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.045053 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.063966 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:09.069227 10578 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:13:09.076684 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.077371 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.085841 10435 raft_consensus.cc:1275] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:13:09.060135 18020 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.068169 18033 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:09.096983 10507 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:09.095474 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.074712 18125 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:09.034206 18123 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:09.088465 18112 consensus_meta.cc:287] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 12:13:09.113951 18112 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:09.099443 10403 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:09.119234 18193 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:09.119771 18193 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:09.120633 18193 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.014086 18195 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.124820 18195 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:09.125702 18206 consensus_queue.cc:1048] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:09.126916 17883 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:09.128224 18091 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:13:09.104058 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.130388 10430 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:13:09.130668 10578 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:13:09.106381 18211 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:13:09.087886 18089 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:09.113137 18123 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.090557 18020 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.099925 18166 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:09.146593 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.148044 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.152010 18123 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.152350 18202 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.152565 18123 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.152953 18202 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.161461 18202 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.168560 11177 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:09.143903 18019 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.005s
W20260709 12:13:09.191406 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.173213 18195 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:09.208182 18020 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:09.156558 18166 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.174504 18019 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:09.228018 18186 log.cc:927] Time spent T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.669s	user 0.002s	sys 0.004s
I20260709 12:13:09.187333 18191 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:13:09.215036 17883 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.187449 18106 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.234843 18195 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.217092 18020 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:09.252614 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:09.253646 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:09.256183 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.251507 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.260022 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.260300 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:09.267719 11213 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:09.240742 18123 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.261793 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:09.274622 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 87560.4us
I20260709 12:13:09.275558 10435 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:13:09.249433 17883 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.280421 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.282675 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.286269 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:09.303800 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 116133us
W20260709 12:13:09.297397 18220 log.cc:927] Time spent T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:13:09.304633 18020 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:09.311458 18106 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:09.311268 18020 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.313647 18020 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.316501 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:09.325490 10507 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:09.325959 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:09.327250 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.266515 18195 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.336612 18195 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:09.304306 18104 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.337720 18104 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.340441 18019 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:09.341627 10580 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:13:09.303584 18123 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.348165 18123 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.318131 18033 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:09.349717 18033 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.350576 18123 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.352120 18033 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:09.386641 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 51331.2us
I20260709 12:13:09.391328 11213 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:09.394948 11217 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:13:09.340451 18104 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.375546 17822 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.404852 10510 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:13:09.406827 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.399442 17822 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.408308 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.410238 17822 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.411085 10437 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:13:09.405381 10510 pending_rounds.cc:85] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting all ops after (but not including) 10
I20260709 12:13:09.413801 10510 pending_rounds.cc:107] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:13:09.414208 10510 raft_consensus.cc:2889] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:09.428035 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.394236 18111 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:13:09.410849 18050 consensus_queue.cc:1059] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:13:09.444444 11213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.446547 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.446720 11213 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:09.450178 11215 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.462589 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.463780 10508 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:09.470539 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:09.471730 10512 raft_consensus.cc:2393] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:09.475939 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:09.479813 10508 raft_consensus.cc:1275] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:13:09.480734 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.475044 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:09.485543 10512 raft_consensus.cc:2368] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:13:09.486346 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.420420 18194 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:13:09.504634 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:09.507148 10432 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 4437) took 1213 ms. Trace:
W20260709 12:13:09.508519 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48703.8us
I20260709 12:13:09.512154 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:13:09.514027 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57173.4us
I20260709 12:13:09.507586 10432 rpcz_store.cc:276] 0709 12:13:08.279362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.279508 (+   146us) service_pool.cc:224] Handling call
0709 12:13:09.453546 (+1174038us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b540012ff54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:09.492657 (+ 39111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324516224}
I20260709 12:13:09.526010 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:09.527434 10587 raft_consensus.cc:2368] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:09.494271 18133 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:09.533352 10584 raft_consensus.cc:1240] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:13:09.538800 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:09.542235 18233 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:09.506606 18020 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:09.545440 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:09.539961 18231 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.158s	user 0.000s	sys 0.008s
I20260709 12:13:09.532948 17903 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:09.534288 18229 raft_consensus.cc:3055] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:13:09.535684 18133 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:09.538587 18165 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:13:09.559010 18166 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:09.555341 18229 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:09.570519 18229 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:09.572417 18229 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:09.550089 18020 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.567689 18133 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.585804 18020 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:09.586858 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:09.587476 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.587682 10433 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.590185 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:09.597482 10433 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:09.556720 17903 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.609606 10509 raft_consensus.cc:1240] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 52bae2d2caf94d97b1049ecd4f82a2df for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:13:09.610769 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.620579 10506 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
I20260709 12:13:09.608925 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.601603 10512 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:13:09.615922 18228 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 12:13:09.615535 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.613772 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.628484 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.626183 11178 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:09.637384 18197 raft_consensus.cc:2804] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:09.638567 17903 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.639956 18104 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:09.647948 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:09.648738 10584 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:09.649111 18197 raft_consensus.cc:697] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:09.650393 18197 consensus_queue.cc:237] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.640384 18155 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:09.641656 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.634122 18125 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:13:09.656407 10479 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:09.659986 17985 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:09.661114 17985 tablet_bootstrap.cc:492] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:13:09.661928 17985 ts_tablet_manager.cc:1403] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 2.293s	user 0.059s	sys 0.030s
I20260709 12:13:09.662480 10584 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 23.
I20260709 12:13:09.663184 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:09.599094 18230 raft_consensus.cc:493] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:09.587792 18055 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:13:09.680413 18230 raft_consensus.cc:515] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.719972 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.721637 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.724545 17883 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.725368 11177 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.727577 18098 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:09.728159 18098 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.728730 18098 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.729619 10582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36746 (request call id 1666) took 1437 ms. Trace:
I20260709 12:13:09.730110 10582 rpcz_store.cc:276] 0709 12:13:08.291692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.291843 (+   151us) service_pool.cc:224] Handling call
0709 12:13:09.729390 (+1437547us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b540010b8d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:09.729537 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493474688}
I20260709 12:13:09.741909 18118 raft_consensus.cc:3055] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:13:09.666808 18054 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:09.745955 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.671021 18155 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:09.667058 17903 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:09.756264 10581 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:13:09.676931 18130 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:09.742763 18118 raft_consensus.cc:740] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:09.762275 18130 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:09.773653 18130 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.810504 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:09.759847 17903 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:09.812637 10504 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:09.831459 18230 leader_election.cc:290] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.839403 18118 consensus_queue.cc:260] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.840761 18118 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:09.845557 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.847654 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:09.848377 11219 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
W20260709 12:13:09.853145 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 127878us
I20260709 12:13:09.857266 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:09.860067 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:09.873589 17903 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:09.877430 18166 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:09.878151 18166 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.878614 18166 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:09.882887 18042 raft_consensus.cc:993] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:09.883746 18042 raft_consensus.cc:1081] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:09.892269 10581 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:09.892709 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:09.893558 10578 raft_consensus.cc:3060] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.897957 18098 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.842702 17985 raft_consensus.cc:359] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:09.903759 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:09.908679 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.910529 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.923820 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:09.930995 18098 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:09.936892 10553 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.936687 11212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57892
I20260709 12:13:09.937553 18242 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:09.937662 11212 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:09.938380 11212 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:09.938387 18242 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:09.940634 18112 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:09.942413 18242 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:09.948367 11221 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:13:09.899801 17985 raft_consensus.cc:740] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:09.949842 17985 consensus_queue.cc:260] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:09.952426 17985 ts_tablet_manager.cc:1434] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.290s	user 0.012s	sys 0.000s
I20260709 12:13:09.954332 18165 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:09.993352 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 236960us
W20260709 12:13:09.993688 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 247824us
I20260709 12:13:09.997010 10478 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:10.005957 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.009496 11219 raft_consensus.cc:1217] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Deduplicated request from leader. Original: 19.11->[19.12-19.12]   Dedup: 19.12->[]
I20260709 12:13:10.030893 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.002501 18155 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:09.954340 18130 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.040977 10434 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48990 (request call id 4440) took 1709 ms. Trace:
I20260709 12:13:09.984273 18165 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.041414 10434 rpcz_store.cc:276] 0709 12:13:08.331769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.332013 (+   244us) service_pool.cc:224] Handling call
0709 12:13:08.383628 (+ 51615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:08.990380 (+606752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.040681 (+1050301us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b540034b054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:13:10.040907 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393788672}
I20260709 12:13:10.040760 18241 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:10.045053 18241 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.034497 18054 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:10.036679 18202 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:10.056701 18118 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:10.057413 18118 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.059218 18130 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.060493 18118 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:10.055893 18155 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.064548 18109 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.065435 18109 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:10.070489 18109 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:10.072188 17872 raft_consensus.cc:1064] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:13:10.073418 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.075713 18241 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.081651 17872 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.078605 18165 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:10.084304 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 49411.6us
I20260709 12:13:10.084578 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 313) took 6916 ms. Trace:
I20260709 12:13:10.095680 10478 leader_election.cc:400] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:10.096117 10478 leader_election.cc:403] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:13:10.112534 11219 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 20. Current term is 21. Ops: []
W20260709 12:13:10.114912 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 353406us
I20260709 12:13:10.116897 10552 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:10.122949 18098 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:10.123974 18098 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.127983 18098 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:10.139083 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
W20260709 12:13:10.143550 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 286383us
I20260709 12:13:10.145072 10434 raft_consensus.cc:1240] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 13. Current term is 14. Ops: []
I20260709 12:13:10.145421 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:13:10.149112 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53696.2us
I20260709 12:13:10.151373 18241 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:10.097119 10336 rpcz_store.cc:276] 0709 12:13:03.168400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.168872 (+   472us) service_pool.cc:224] Handling call
0709 12:13:03.452614 (+283742us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a49090 00007f2c92ae7f3a 00007f2c9889de9c 00007f2c9899c04e 00007f2c96e5d2a4 00007f2c96e5d232 00007f2c96e5d161 00007f2c96e5d0dc 00007f2c96e5c392 00007f2c90e1a48c 00007f2c90e198c6
0709 12:13:10.084472 (+6631858us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:07.562137 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:07.762947 (+200810us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:07.765838 (+  2891us) write_op.cc:435] Acquired schema lock
0709 12:13:07.773779 (+  7941us) tablet.cc:662] Decoding operations
0709 12:13:07.834701 (+ 60922us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:07.841431 (+  6730us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:07.846840 (+  5409us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:13:07.963644 (+116804us) tablet.cc:701] Row locks acquired
0709 12:13:07.968420 (+  4776us) write_op.cc:260] PREPARE: finished
0709 12:13:07.969997 (+  1577us) write_op.cc:270] Start()
0709 12:13:07.970153 (+   156us) write_op.cc:276] Timestamp: P: 1783599187969933 usec, L: 0
0709 12:13:07.970177 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:13:07.970968 (+   791us) log.cc:844] Serialized 907 byte log entry
0709 12:13:08.138765 (+167797us) op_driver.cc:464] REPLICATION: finished
0709 12:13:08.518513 (+379748us) write_op.cc:301] APPLY: starting
0709 12:13:08.520363 (+  1850us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:08.715786 (+195423us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:08.715912 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:08.841340 (+125428us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:08.855979 (+ 14639us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:13:08.858326 (+  2347us) write_op.cc:312] APPLY: finished
0709 12:13:08.877197 (+ 18871us) log.cc:844] Serialized 33 byte log entry
0709 12:13:08.885314 (+  8117us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:08.891136 (+  5822us) write_op.cc:454] Released schema lock
0709 12:13:08.891187 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:08.891721 (+   534us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:08.891766 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":250031,"reported_tablets":10,"spinlock_wait_cycles":348249856,"child_traces":[["op",{"apply.queue_time_us":358630,"apply.run_cpu_time_us":9190,"apply.run_wall_time_us":384376,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":194583,"prepare.run_cpu_time_us":22027,"prepare.run_wall_time_us":493333,"raft.queue_time_us":46754,"raft.run_cpu_time_us":18970,"raft.run_wall_time_us":205013,"replication_time_us":153518,"thread_start_us":306246,"threads_started":4,"wal-append.queue_time_us":88239,"wal-append.run_cpu_time_us":17601,"wal-append.run_wall_time_us":1823811}]]}
W20260709 12:13:10.167399 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.175287 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.176297 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
W20260709 12:13:10.169576 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 52805.1us
I20260709 12:13:10.133131 18109 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21.
I20260709 12:13:10.178440 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.180743 10578 raft_consensus.cc:2468] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:10.195457 17903 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:10.196235 17903 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:10.196830 17903 raft_consensus.cc:3060] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:10.188381 18133 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:10.204954 10582 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:13:10.190847 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:10.191097 18166 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.168632 11212 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.171655 18241 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:10.162765 10332 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 16, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:10.194109 18240 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:10.209928 10332 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 11 to 12, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 12 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:10.180809 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:10.208191 11221 raft_consensus.cc:1240] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:10.212132 10552 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:10.211717 10332 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 14 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 15 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:10.212989 17778 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:10.213581 18042 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:13:10.207103 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 202771us
I20260709 12:13:10.210515 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:10.165480 18098 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:10.219391 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.216881 10332 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 19 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:10.220465 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.221611 10478 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.225771 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:10.226001 10552 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:10.208873 18020 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:10.220455 10479 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:10.260515 17883 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:10.294461 10581 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:10.295446 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:10.285081 18033 consensus_queue.cc:1059] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:13:10.298277 17883 raft_consensus.cc:3055] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:10.298789 17883 raft_consensus.cc:740] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:10.299933 17883 consensus_queue.cc:260] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.301371 17883 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:10.224017 18198 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.302225 18198 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:10.260107 18020 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:10.233073 18030 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.303690 18020 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:10.304248 18030 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.305812 18198 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:10.280234 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 59916us
W20260709 12:13:10.279527 11176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.260063 10509 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:10.231572 11212 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:10.243875 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:10.309226 10511 raft_consensus.cc:606] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:10.309613 10511 raft_consensus.cc:614] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:10.256651 10510 raft_consensus.cc:1240] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:10.257508 10505 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:10.258751 10552 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:10.308252 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:10.311757 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 53129.9us
I20260709 12:13:10.316574 10552 leader_election.cc:400] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:13:10.316905 10552 leader_election.cc:403] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:13:10.318048 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 72615.1us
I20260709 12:13:10.217780 18104 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:10.324684 10552 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:10.323199 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 57977.7us
I20260709 12:13:10.215380 18241 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:10.232851 18043 raft_consensus.cc:2764] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:13:10.338081 18221 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:13:10.345043 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.281373 18130 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:10.322935 18033 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:10.328117 18073 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:10.338466 18242 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:10.328274 18019 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:10.232863 18165 consensus_queue.cc:1048] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.114s
W20260709 12:13:10.479732 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 155182us
I20260709 12:13:10.224380 18240 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.355379 18104 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
W20260709 12:13:10.463421 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:10.466231 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 109552us
I20260709 12:13:10.468089 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:10.276507 18233 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:13:10.341707 18221 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.354185 18195 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.126s
I20260709 12:13:10.356345 18112 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 12:13:10.506721 17870 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:10.507763 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:13:10.509042 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 144519us
I20260709 12:13:10.573670 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:10.601477 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:10.613157 18130 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.659773 10511 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 21: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
W20260709 12:13:10.592176 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 105683us
I20260709 12:13:10.660989 17870 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.664646 17870 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.665963 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.667019 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.667460 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.669860 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.671769 10513 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:13:10.480103 18019 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:10.466421 18030 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:10.587512 18245 log.cc:927] Time spent T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:13:10.632062 18089 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:13:10.632246 18226 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.678388 18226 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
W20260709 12:13:10.680079 18260 log.cc:927] Time spent T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:13:10.681336 18226 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:10.478706 18033 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:10.556901 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.683828 11214 raft_consensus.cc:2410] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:13:10.466899 18197 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:10.685159 18197 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.687227 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.695462 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:10.592401 18254 log.cc:927] Time spent T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
W20260709 12:13:10.621598 18257 log.cc:927] Time spent T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:13:10.647943 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.592912 18166 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.654686 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.652920 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-945dd7b821a5493f8a9d6ca251795da9 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:13:10.710656 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:10.711457 10505 raft_consensus.cc:3060] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:10.715035 11214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:10.715128 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.715927 11214 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:10.688401 18197 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:10.724457 10551 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.728121 18019 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:10.745205 10505 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 23.
W20260709 12:13:10.747514 18258 log.cc:927] Time spent T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 12:13:10.758565 11214 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 24.
I20260709 12:13:10.716441 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-945dd7b821a5493f8a9d6ca251795da9 on tablet 945dd7b821a5493f8a9d6ca251795da9 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:10.764714 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8b08b19bdbb4ac7b6bae9eccebb6335" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.766388 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.767342 10585 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:13:10.774976 18266 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.776031 18266 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:10.776729 18221 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.652616 18104 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:10.782483 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.789057 10552 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:10.791368 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 53378.1us
W20260709 12:13:10.804665 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 85328.7us
I20260709 12:13:10.816448 11221 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:13:10.818343 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 160026us
I20260709 12:13:10.819651 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.820457 11221 raft_consensus.cc:2368] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 23. Current term is 24.
I20260709 12:13:10.700043 18130 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.822844 11214 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:10.823359 11177 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:10.832468 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 43532.9us
I20260709 12:13:10.833263 10404 leader_election.cc:400] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 23. Current term is 24.
I20260709 12:13:10.833657 10404 leader_election.cc:403] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:10.835371 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:10.833482 10553 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:10.846096 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.841463 11177 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:10.848968 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.849665 10432 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
I20260709 12:13:10.853462 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.855876 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.858683 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.859391 10432 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
I20260709 12:13:10.861631 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:10.839604 18238 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:10.836809 18165 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:10.864645 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.827180 18253 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:13:10.843320 18043 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:10.896252 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.896981 10431 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
I20260709 12:13:10.898249 10431 raft_consensus.cc:1275] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:13:10.852738 18266 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:10.873984 10437 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:10.861193 17822 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 12:13:10.901371 18256 log.cc:927] Time spent T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.435s	user 0.006s	sys 0.000s
I20260709 12:13:10.843621 18166 raft_consensus.cc:2804] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:10.878250 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:10.878476 10432 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:13:10.879572 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.880483 10436 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:10.905145 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.881253 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:10.883877 18020 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.579s	user 0.017s	sys 0.000s
I20260709 12:13:10.884994 18130 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.885504 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:10.890002 17883 consensus_meta.cc:287] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.588s	user 0.013s	sys 0.000s
I20260709 12:13:10.890794 18240 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:10.891956 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.940876 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:10.941392 10512 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 17.
I20260709 12:13:10.861836 18118 raft_consensus.cc:2804] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:10.906239 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:10.906458 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:10.883414 18262 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:13:10.908329 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:10.849283 18198 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:13:10.952306 10478 leader_election.cc:304] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:10.855072 18129 raft_consensus.cc:3060] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:13:10.941083 18020 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:10.921571 11221 raft_consensus.cc:1240] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:10.905225 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.910720 10436 raft_consensus.cc:606] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:10.958437 10578 raft_consensus.cc:2393] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 19: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:10.950152 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:13:10.947826 18091 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:13:10.942018 17822 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:10.925462 17778 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:10.963984 18118 raft_consensus.cc:697] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
W20260709 12:13:10.972949 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 90208.9us
I20260709 12:13:10.985630 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:10.963292 18050 raft_consensus.cc:2749] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:10.964809 18133 consensus_meta.cc:287] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.006s
W20260709 12:13:10.972334 17903 consensus_meta.cc:287] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.775s	user 0.013s	sys 0.005s
W20260709 12:13:10.970817 18261 log.cc:927] Time spent T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 12:13:11.005446 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:11.007480 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:11.001240 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.013547 10582 raft_consensus.cc:2468] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 23.
I20260709 12:13:10.969730 18048 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:10.984212 18130 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:10.956985 18030 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
W20260709 12:13:11.057348 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 44770.3us
I20260709 12:13:11.034335 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.060767 10553 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.035055 18266 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:13:11.000545 18242 consensus_meta.cc:287] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
I20260709 12:13:11.022197 10433 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
W20260709 12:13:11.046391 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48020.1us
I20260709 12:13:10.972421 18068 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:11.039305 18048 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.077009 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:11.040234 18271 consensus_queue.cc:1059] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:13:11.085747 18112 raft_consensus.cc:3055] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:11.085847 18166 raft_consensus.cc:697] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:11.086199 18112 raft_consensus.cc:740] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:11.086931 18166 consensus_queue.cc:237] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.087131 18112 consensus_queue.cc:260] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.088441 18112 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:11.094427 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:10.995031 17778 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.047374 18130 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:11.070464 18241 consensus_meta.cc:287] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.005s
I20260709 12:13:11.006207 18233 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:13:11.006556 18089 consensus_queue.cc:1059] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:13:11.121759 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:11.076016 18242 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
W20260709 12:13:11.099675 18091 log.cc:927] Time spent T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.054s	user 0.002s	sys 0.001s
I20260709 12:13:11.040023 17903 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:11.124642 18234 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:13:11.124132 18130 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:11.093652 18280 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:11.130465 18033 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:11.117583 18020 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:11.137357 18268 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:11.083889 18068 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.141657 18271 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:11.069101 18118 consensus_queue.cc:237] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.121655 18266 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:11.138019 18133 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:11.128180 18241 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.243794 17778 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.214437 18020 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:11.261773 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 201136us
I20260709 12:13:11.262854 10553 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:11.241192 18091 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:13:11.256479 10583 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
W20260709 12:13:11.251300 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 129658us
I20260709 12:13:11.215552 18033 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:11.272675 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 150336us
I20260709 12:13:11.206324 18098 consensus_queue.cc:1048] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:13:11.275104 11178 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.199090 18280 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.289835 18269 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:11.291910 18269 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.292389 18269 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:11.301482 18271 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:11.302054 18271 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:11.283532 18195 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:11.303097 18269 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.272799 18020 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.304700 17903 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:11.305425 18269 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.306227 18020 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.310238 18129 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987) with higher term. Message: Invalid argument: T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for earlier term 23. Current term is 24.
I20260709 12:13:11.316298 18271 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:11.317317 10578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36746 (request call id 1682) took 1114 ms. Trace:
I20260709 12:13:11.279659 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:11.319881 18091 tablet_bootstrap.cc:492] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:13:11.321731 18241 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.326009 18280 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.278421 18033 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:11.332074 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 377850us
I20260709 12:13:11.360488 10478 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:11.317746 10581 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:11.362284 18030 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:11.337170 10578 rpcz_store.cc:276] 0709 12:13:10.202806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.203106 (+   300us) service_pool.cc:224] Handling call
0709 12:13:11.317060 (+1113954us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400256a54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:11.317239 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697204480}
I20260709 12:13:11.364496 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:11.356068 10430 raft_consensus.cc:2468] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 24.
I20260709 12:13:11.379385 10509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c882875c377649acb233035d47b8132e"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:11.380333 10509 raft_consensus.cc:606] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:11.380820 10509 raft_consensus.cc:614] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:11.364892 18129 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.377653 18091 ts_tablet_manager.cc:1403] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 2.250s	user 0.054s	sys 0.025s
I20260709 12:13:11.374495 18195 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:11.384346 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 300597us
I20260709 12:13:11.388757 18068 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.439073  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:11.442253  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:11.450181  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:11.387338 18129 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.465435 18073 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.491947 18073 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:11.504978 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.506129 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.471698 18292 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:11.511623 10431 raft_consensus.cc:2393] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 22: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:11.506202 10437 raft_consensus.cc:2393] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 22: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:11.534763 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:11.535640 10504 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:11.515187 18073 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.513512 18196 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:11.541618 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 175195us
I20260709 12:13:11.542080 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.547513 10437 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:13:11.530975 18271 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.497928 18091 raft_consensus.cc:359] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.554121 18091 raft_consensus.cc:740] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:11.555025 18091 consensus_queue.cc:260] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:11.556124 10504 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 18.
I20260709 12:13:11.557610 10403 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:11.520722 18292 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.562034 18243 raft_consensus.cc:3055] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:11.562620 18243 raft_consensus.cc:740] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:11.563988 18243 consensus_queue.cc:260] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.564699 18048 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.565925 18243 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:11.560151 18033 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:11.577191 17870 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:11.577903 17870 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
W20260709 12:13:11.578408 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 557578us
I20260709 12:13:11.551342 18196 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.583897 18305 consensus_queue.cc:1059] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:11.585798 18305 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:11.586356 18305 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:11.587299 18292 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:11.588469 18196 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.589800 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:11.590509 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:11.591367 10509 raft_consensus.cc:2393] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:11.591933 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:11.591189 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:11.592675 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:11.622268 17870 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:11.593115 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:11.659315 10510 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:11.657557 10512 raft_consensus.cc:2393] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 18: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:11.624204 18291 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
W20260709 12:13:11.662160 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 382615us
I20260709 12:13:11.618639 10433 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48990 (request call id 4465) took 1716 ms. Trace:
I20260709 12:13:11.663551 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:11.675804 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:11.676761 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0bb77704fc9403481da02083315dbb1" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:11.616051 18130 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.620743 18129 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.683267 18291 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.685721 18291 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.666714 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.669091 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:13:11.671252 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 124682us
I20260709 12:13:11.690662 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.673573 10433 rpcz_store.cc:276] 0709 12:13:09.902132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.903385 (+  1253us) service_pool.cc:224] Handling call
0709 12:13:11.618405 (+1715020us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540012ff54. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:11.618560 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2569105152}
I20260709 12:13:11.692548 10510 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
I20260709 12:13:11.705981 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:11.707162 10582 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 23.
I20260709 12:13:11.708833 10586 raft_consensus.cc:1240] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:11.711360 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.712123 10430 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:11.560719 18197 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:11.713274 18197 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.714326 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:11.723132 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:11.694371 18263 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:11.617825 18289 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:11.727066 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ecefe266eea48c4af129b425f391fe9" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:11.728607 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:11.730921 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:11.731423 10553 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:11.731742 10582 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:11.732311 11221 raft_consensus.cc:2468] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:13:11.731611 10430 raft_consensus.cc:3060] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:11.741812 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:11.736186 10586 raft_consensus.cc:2368] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 22. Current term is 23.
I20260709 12:13:11.748729 10578 raft_consensus.cc:1275] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:11.785851 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:11.756114 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:11.746434 18109 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.746990 18165 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:11.766194 10552 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:11.800496 18197 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.802923 10478 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.803161 18109 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.805534 18109 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.770840 10479 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:11.772413 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 3926) took 1083 ms. Trace:
I20260709 12:13:11.750998 18292 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:11.696071 18280 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:11.791783 18268 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:11.792451 10553 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:11.793601 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.794242 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.794370 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:11.729715 18289 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.832165 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:11.834020 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:11.834966 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64223.3us
I20260709 12:13:11.836381 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:11.825812 18300 log.cc:927] Time spent T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:13:11.806092 18109 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:11.811810 10508 rpcz_store.cc:276] 0709 12:13:10.688825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.689707 (+   882us) service_pool.cc:224] Handling call
0709 12:13:10.690023 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:10.854808 (+164785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.854840 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:10.860808 (+  5968us) log.cc:844] Serialized 52 byte log entry
0709 12:13:10.974244 (+113436us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:11.247383 (+273139us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.256849 (+  9466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.769797 (+512948us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54000da054. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adfe01 00007f2c92ade83e 00007f2c97aa3419 00007f2c9101abd4 00007f2c9101ab62 00007f2c9101aa91 00007f2c9101aa0c 00007f2c91019cc2 00007f2c90e1a48c 00007f2c90e198c6
0709 12:13:11.769871 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.771979 (+  2108us) raft_consensus.cc:1685] finished
0709 12:13:11.772127 (+   148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.772298 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20900,"prepare.queue_time_us":41293,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":27088,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":13849,"spinlock_wait_cycles":338793728,"thread_start_us":335885,"threads_started":2,"wal-append.queue_time_us":144998}
I20260709 12:13:11.833817 18246 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:13:11.838840 18073 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:13:11.834288 18289 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:11.806279 18160 consensus_queue.cc:1059] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:13:11.863722 18160 raft_consensus.cc:3055] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:11.864108 18160 raft_consensus.cc:740] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:11.865064 18160 consensus_queue.cc:260] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.830709 18292 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.866812 18160 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:11.868709 18292 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:11.738118 18263 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.872062 18263 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:11.872502 18130 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:11.875905 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.876420 10429 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:11.845386 10478 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:11.846920 18042 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:11.849584 18073 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:11.896198 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50927.3us
I20260709 12:13:11.855175 17778 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:11.857403 18306 raft_consensus.cc:1064] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:13:11.836360 18281 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:11.836513 17822 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:11.838775 18269 raft_consensus.cc:2804] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:11.906277 17822 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.906785 18269 raft_consensus.cc:697] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:11.907851 18269 consensus_queue.cc:237] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.908672 17822 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:11.830716 18268 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.870550 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:11.737305 18305 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.826175 18165 raft_consensus.cc:2804] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:11.880385 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:11.848775 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:11.884667 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 45162.1us
I20260709 12:13:11.879002 18042 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:11.892920 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:11.893878 18246 raft_consensus.cc:3055] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:11.902881 10478 leader_election.cc:400] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 22. Current term is 23.
I20260709 12:13:11.897796 18073 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:11.905736 18046 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:11.910146 10437 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:11.912287 18268 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:11.911979 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:11.939034 18285 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:11.939463 18285 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.939785 18285 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:11.909338 17822 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:13:11.943674 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:11.945933 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.946595 10583 raft_consensus.cc:1240] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 16. Current term is 17. Ops: []
I20260709 12:13:11.916260 10430 raft_consensus.cc:2468] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
W20260709 12:13:11.916435 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 67754.1us
I20260709 12:13:11.918082 18253 consensus_queue.cc:1048] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:13:11.918972 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.920428 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:11.974130 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.920790 18241 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:11.920882 18091 ts_tablet_manager.cc:1434] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.504s	user 0.018s	sys 0.015s
I20260709 12:13:11.835143 18280 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:11.928951 10478 leader_election.cc:403] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:11.908864 18306 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:11.980520 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 79204us
I20260709 12:13:11.983233 10438 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:13:11.983213 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:11.925509 18042 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:11.986785 10437 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 22.
I20260709 12:13:11.984253 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:11.926594 18246 raft_consensus.cc:740] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:11.925776 18165 raft_consensus.cc:697] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:12.023820 18165 consensus_queue.cc:237] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:11.997613 17870 consensus_queue.cc:1059] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:11.978711 17778 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:13:12.001469 18020 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:12.002051 18241 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:12.004673 18280 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:12.005726 10580 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 13.
W20260709 12:13:11.977347 18091 ts_tablet_manager.cc:1467] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Tablet startup took 586ms
W20260709 12:13:12.034300 18091 ts_tablet_manager.cc:1469] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Trace:
0709 12:13:05.246676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.250699 (+  4023us) service_pool.cc:224] Handling call
0709 12:13:05.411412 (+160713us) ts_tablet_manager.cc:934] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
0709 12:13:06.980891 (+1569479us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:07.959331 (+978440us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:07.959480 (+   149us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:08.174091 (+214611us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:08.174468 (+   377us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:08.174820 (+   352us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:08.174888 (+    68us) inbound_call.cc:177] Queueing success response
0709 12:13:09.127850 (+952962us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:13:11.417365 (+2289515us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:13:11.467400 (+ 50035us) tablet_replica.cc:261] Starting instrumentation
0709 12:13:11.468470 (+  1070us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":32587,"raft.queue_time_us":23890,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":214,"tablet-copy.queue_time_us":139625,"tablet-download-pool.queue_time_us":1834,"tablet-download-pool.run_cpu_time_us":1721,"tablet-download-pool.run_wall_time_us":527376,"thread_start_us":306035,"threads_started":3}
I20260709 12:13:11.991137 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:12.039228 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48203.3us
I20260709 12:13:12.010746 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:12.016170 10587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55466 (request call id 3670) took 1111 ms. Trace:
I20260709 12:13:11.984992 10430 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:11.988308 18030 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 22. Current term is 23.
I20260709 12:13:12.045542 10438 raft_consensus.cc:2393] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 14: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:12.042707 10587 rpcz_store.cc:276] 0709 12:13:10.904839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.904966 (+   127us) service_pool.cc:224] Handling call
0709 12:13:12.013556 (+1108590us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400213254. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:12.016089 (+  2533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712229120}
I20260709 12:13:12.053417 18280 consensus_queue.cc:1059] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:12.056841 18280 consensus_queue.cc:1243] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:13:12.061545 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:12.062865 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:12.048561 10479 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:12.057375 18280 raft_consensus.cc:3055] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:13:11.911854 18281 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:11.934851 18289 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.067957 18280 raft_consensus.cc:740] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:12.070088 11177 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:12.066018 18105 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:12.071966 18280 consensus_queue.cc:260] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:12.079882 18280 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:13:12.084964 18195 consensus_meta.cc:287] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.002s
I20260709 12:13:12.085532 18195 raft_consensus.cc:2749] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:12.086344 11219 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:12.092422 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 43994.4us
I20260709 12:13:12.092526 18230 raft_consensus.cc:2804] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:12.095382 18230 raft_consensus.cc:697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:12.096293 18230 consensus_queue.cc:237] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.101450 18098 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:12.105314 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 293) took 8125 ms. Trace:
I20260709 12:13:12.106153 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:12.107507 18238 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:12.108180 18089 raft_consensus.cc:993] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:12.108721 18089 raft_consensus.cc:1081] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:12.150671 10504 raft_consensus.cc:1240] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:13:12.114099 18285 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.118392 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:12.129626 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:12.117813 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:12.171968 18285 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:12.156152 17903 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.114871 10587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55466
I20260709 12:13:12.173547 10587 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:12.173980 10587 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:12.173398 17903 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:12.153350 10506 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:13:12.143646 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:12.171823 18073 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:12.146960 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:12.177266 18303 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:12.178825 18073 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:12.179603 10512 raft_consensus.cc:2368] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:13:12.136358 18289 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:12.163646 18246 consensus_queue.cc:260] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.116707 18281 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:12.182119 18268 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.148496 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:12.137466 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:12.136703 10579 raft_consensus.cc:2368] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 22. Current term is 23.
I20260709 12:13:12.194434 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:12.194967 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:12.226738 10585 raft_consensus.cc:3060] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.134229 10328 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:12.150010 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-26d3d17d58be40c8974c348ca146a951 received from {username='slave'} at 127.0.0.1:38166
I20260709 12:13:12.202080 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:12.210439 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:12.212260 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:12.210044 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:12.216140 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:12.229502 10583 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:13:12.229578 10431 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:12.217681 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 63853.5us
I20260709 12:13:12.233135 18288 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:12.234799 11218 raft_consensus.cc:1275] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:13:12.234200 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:12.238481 18033 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:12.121562 18305 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:12.244426 10587 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:12.258882 10433 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:12.245628 10328 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index -1 to 12, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:12.267565 18241 consensus_queue.cc:1048] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:12.235257 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-26d3d17d58be40c8974c348ca146a951 on tablet 26d3d17d58be40c8974c348ca146a951 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:12.269485 17903 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:12.272665 18268 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:12.274413 10552 leader_election.cc:400] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
I20260709 12:13:12.274775 10552 leader_election.cc:403] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.276764 10513 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:12.278107 18138 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:12.280426 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:12.281381 10505 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:13:12.281950 10586 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:12.281224 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:12.288342 18089 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:12.288369 10478 leader_election.cc:400] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 22. Current term is 23.
I20260709 12:13:12.289723 10478 leader_election.cc:403] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.291141 18073 raft_consensus.cc:3060] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:12.294538 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:12.296005 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:12.304193 10581 raft_consensus.cc:1275] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:13:12.105772 10332 rpcz_store.cc:276] 0709 12:13:03.979893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.980946 (+  1053us) service_pool.cc:224] Handling call
0709 12:13:12.105238 (+8124292us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:10.612082 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:10.755520 (+143438us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:10.755593 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:13:10.755632 (+    39us) tablet.cc:662] Decoding operations
0709 12:13:10.756733 (+  1101us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:10.756849 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:10.756889 (+    40us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:13:10.757511 (+   622us) tablet.cc:701] Row locks acquired
0709 12:13:10.757540 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:13:10.757745 (+   205us) write_op.cc:270] Start()
0709 12:13:10.757916 (+   171us) write_op.cc:276] Timestamp: P: 1783599190757697 usec, L: 0
0709 12:13:10.757952 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:13:10.792295 (+ 34343us) log.cc:844] Serialized 1677 byte log entry
0709 12:13:11.495414 (+703119us) op_driver.cc:464] REPLICATION: finished
0709 12:13:11.729992 (+234578us) write_op.cc:301] APPLY: starting
0709 12:13:11.747200 (+ 17208us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:11.806362 (+ 59162us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:11.806497 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:11.820124 (+ 13627us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:11.820364 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:13:11.820406 (+    42us) write_op.cc:312] APPLY: finished
0709 12:13:11.820725 (+   319us) log.cc:844] Serialized 49 byte log entry
0709 12:13:11.820941 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:11.821068 (+   127us) write_op.cc:454] Released schema lock
0709 12:13:11.821114 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:11.846808 (+ 25694us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:11.850841 (+  4033us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":180684,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":219851,"apply.run_cpu_time_us":9549,"apply.run_wall_time_us":117188,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":238669,"prepare.run_cpu_time_us":55070,"prepare.run_wall_time_us":368323,"raft.queue_time_us":430687,"raft.run_cpu_time_us":8917,"raft.run_wall_time_us":70471,"replication_time_us":737210,"thread_start_us":698034,"threads_started":4,"wal-append.queue_time_us":200569}]]}
I20260709 12:13:12.306912 10579 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:12.309582 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:12.311916 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:12.312618 10430 raft_consensus.cc:3055] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:13:12.313055 10430 raft_consensus.cc:740] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:12.315215 10587 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:12.325577 10585 raft_consensus.cc:2468] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 22.
I20260709 12:13:12.311483 10438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:12.328286 10438 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:12.335430 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:12.322863 10583 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:13:12.335433 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:12.339159 10429 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:12.296660 10328 catalog_manager.cc:5697] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 11 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:12.338023 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:12.339717 10478 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:12.343336 10328 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:12.344041 10430 consensus_queue.cc:260] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:12.344241 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:12.345139 11219 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:12.357414 10438 raft_consensus.cc:2468] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 15.
I20260709 12:13:12.397253 10430 raft_consensus.cc:3060] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:12.407881 18105 raft_consensus.cc:2804] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:13:12.351452 18269 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:12.330876 18160 raft_consensus.cc:493] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.412952 18160 raft_consensus.cc:515] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:12.343097 18280 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 14. Current term is 15.
W20260709 12:13:12.423727 10404 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.444636 10403 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.450695 18042 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:12.463637 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104036us
I20260709 12:13:12.468950 18285 consensus_queue.cc:1059] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:12.472991 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:12.474192 10508 raft_consensus.cc:2368] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:13:12.480774 18246 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:12.488968 10433 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:12.365931 18306 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:13:12.492107 11216 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:12.482826 18285 consensus_queue.cc:1243] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:12.503343 18068 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:12.330652 18098 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.158s
I20260709 12:13:12.504161 18068 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:12.506588 18068 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.507172 18196 consensus_queue.cc:1048] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:12.510365 18068 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:12.487479 18105 raft_consensus.cc:697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:12.533543 11216 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:13:12.510936 10431 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
I20260709 12:13:12.537554 18269 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:12.538692 10505 raft_consensus.cc:1275] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:13:12.542927 10429 raft_consensus.cc:2468] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:12.556025 10479 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:12.544282 10430 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:13:12.557277 18160 leader_election.cc:290] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:12.560870 18104 raft_consensus.cc:2804] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:12.561797 18104 raft_consensus.cc:697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:12.564146 18104 consensus_queue.cc:237] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.529875 11219 raft_consensus.cc:2468] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:13:12.514009 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:12.567752 10553 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:13:12.526793 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.612222 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 98337.2us
W20260709 12:13:12.612149 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 44922.8us
I20260709 12:13:12.525496 18306 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
W20260709 12:13:12.634289 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48711.2us
I20260709 12:13:12.637081 10552 leader_election.cc:400] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:13:12.640705 10552 leader_election.cc:403] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:13:12.334172 18268 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.298s
I20260709 12:13:12.454788 18042 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:12.360608 18291 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:12.611464 17903 raft_consensus.cc:2804] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:13:12.656752 10553 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:13:12.593978 18162 consensus_queue.cc:1175] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "33d77e3063684586a737bccf358786d8" responder_term: 23 status { last_received { term: 23 index: 13 } last_committed_idx: 12 last_received_current_leader { term: 23 index: 13 } }
I20260709 12:13:12.513619 18105 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:12.673308 10509 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:12.637820 18246 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:12.678865 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:12.682001 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:12.683192 10437 raft_consensus.cc:2368] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:13:12.685279 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bea092b9128d46b6a42f4fe9aa23c739" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:12.671449 18241 raft_consensus.cc:3060] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:12.669958 18068 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:13:12.702913 18198 raft_consensus.cc:2804] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:12.687067 18280 consensus_queue.cc:1048] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:13:12.718520 17903 raft_consensus.cc:697] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:12.711628 18020 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:13:12.772382 18198 raft_consensus.cc:697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
W20260709 12:13:12.827101 18320 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 12:13:12.885403 18073 consensus_meta.cc:287] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.003s
W20260709 12:13:12.906239 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 239159us
W20260709 12:13:12.957666 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 274456us
I20260709 12:13:12.874382 18198 consensus_queue.cc:237] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:12.870807 17903 consensus_queue.cc:237] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:12.906916 18073 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 22. Current term is 23.
W20260709 12:13:12.979302 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 334083us
W20260709 12:13:12.959833 18326 log.cc:927] Time spent T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.182s	user 0.001s	sys 0.001s
I20260709 12:13:12.996784 10553 leader_election.cc:400] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Vote denied by peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
W20260709 12:13:12.999809 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 304358us
I20260709 12:13:13.032505 18238 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:13.090454 18325 log.cc:927] Time spent T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.099s	user 0.001s	sys 0.001s
I20260709 12:13:13.135993 18330 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.096712 18253 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.083915 18238 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:13.145454 10477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.150111 18105 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.138444 18330 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:13.177546 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 95238.3us
W20260709 12:13:13.181497 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 159224us
I20260709 12:13:13.200714 10513 raft_consensus.cc:2468] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
I20260709 12:13:13.178200 18105 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.152192 18253 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:13.201779 18329 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:13:13.200488 18302 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:13.219149 18241 consensus_meta.cc:287] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: Time spent flushing consensus metadata: real 0.502s	user 0.015s	sys 0.001s
W20260709 12:13:13.239280 18328 log.cc:927] Time spent T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:13:13.251626 18105 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.246604 18241 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681) with higher term. Message: Invalid argument: T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for earlier term 19. Current term is 20.
I20260709 12:13:13.246237 18302 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:13.257676 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 51086.8us
W20260709 12:13:13.266036 18341 log.cc:927] Time spent T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 12:13:13.268491 18342 log.cc:927] Time spent T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 12:13:13.269760 18337 log.cc:927] Time spent T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:13:13.272758 18330 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:13.276297 18238 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:13.277537 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:13.278507 10578 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:13.283052 18253 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.297109 18302 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:13.351436 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 92615.4us
I20260709 12:13:13.356845 18238 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.357836 18238 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.363766 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:13.365846 18238 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:13.389086 18362 log.cc:927] Time spent T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:13:13.395651 18193 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.405892 18193 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.416450 18193 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.491518 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.492388 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.489670 18351 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:13.493186 11216 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 20.
I20260709 12:13:13.493376 18351 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.498896 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:13.508744 10578 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:13.471195 18229 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:13.507362 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.510919 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.511933 11216 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:13.512035 10507 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:13.512645 10434 raft_consensus.cc:1275] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:13.512811 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.513236 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.513441 10431 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:13.513880 10436 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
I20260709 12:13:13.492389 10505 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:13.519635 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.519835 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.520356 11219 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:13.520530 11221 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:13:13.523185 18229 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.535833 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.536784 10507 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:13.539114 10507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.534112 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.543320 10509 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:13.516614 17903 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.551167 17903 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:13.566545 18229 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:13.580698 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:13.583171 10578 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:13.585888 18351 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:13.591127 10579 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:13:13.591959 18241 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.615803 18241 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.617769 18241 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:13.619992 17903 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:13.658622 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 147524us
W20260709 12:13:13.671736 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 127376us
I20260709 12:13:13.694144 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.687314 10479 leader_election.cc:304] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:13.733637 10513 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41084 (request call id 3948) took 1454 ms. Trace:
I20260709 12:13:13.746048 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.747748 10509 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:13.749792 11178 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:13.750713 18162 raft_consensus.cc:2749] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:13.751726 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 64854.1us
I20260709 12:13:13.756580 18373 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.758026 18373 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.761739 18373 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:13.768875 18165 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.772017 18198 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.772451 18288 consensus_queue.cc:1048] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:13.773128 18198 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.776662 18291 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:13.777391 18291 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.777994 18291 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:13.781620 10552 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:13.783540 18289 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:13.784034 18289 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.784520 18289 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:13.793282 10404 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:13.794799 18198 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.772409 18165 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:13.799690 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46640.8us
I20260709 12:13:13.714890  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:13.828547 11216 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:13:13.734652 10513 rpcz_store.cc:276] 0709 12:13:12.279318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.295726 (+ 16408us) service_pool.cc:224] Handling call
0709 12:13:13.732911 (+1437185us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540039f3d4. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92ae32fa 00007f2c97aa4b17 00007f2c9101f6d4 00007f2c9101f662 00007f2c9101f591 00007f2c9101f50c 00007f2c9101e7c2 00007f2c90e1a48c 00007f2c90e198c6 00007f2c90e1926f
0709 12:13:13.733290 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580754304}
I20260709 12:13:13.841688 18288 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.842831 18288 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.843099 18020 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:13.843156 18346 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.844538 18372 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.852684 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.855376 18372 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.856441 10508 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:13.857916 18346 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:13.869856 18288 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:13.871120 18346 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:13.873471 18372 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:13.879354 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:13.883181 17870 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:13.883844 17870 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.884295 17870 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:13.884826 10328 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 302) took 6106 ms. Trace:
I20260709 12:13:13.889113 10335 catalog_manager.cc:5697] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.890931 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:13.894104 18373 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.897769 18288 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.903990 18288 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.908241 18288 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:13.908010 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.885308 10328 rpcz_store.cc:276] 0709 12:13:07.778098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.785513 (+  7415us) service_pool.cc:224] Handling call
0709 12:13:13.884455 (+6098942us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:12.740686 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:12.861117 (+120431us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:12.862613 (+  1496us) write_op.cc:435] Acquired schema lock
0709 12:13:12.863739 (+  1126us) tablet.cc:662] Decoding operations
0709 12:13:12.937056 (+ 73317us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:12.967478 (+ 30422us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:12.968276 (+   798us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:13:13.030582 (+ 62306us) tablet.cc:701] Row locks acquired
0709 12:13:13.032130 (+  1548us) write_op.cc:260] PREPARE: finished
0709 12:13:13.048052 (+ 15922us) write_op.cc:270] Start()
0709 12:13:13.053280 (+  5228us) write_op.cc:276] Timestamp: P: 1783599193047938 usec, L: 0
0709 12:13:13.060052 (+  6772us) op_driver.cc:348] REPLICATION: starting
0709 12:13:13.150863 (+ 90811us) log.cc:844] Serialized 1794 byte log entry
0709 12:13:13.328278 (+177415us) op_driver.cc:464] REPLICATION: finished
0709 12:13:13.374379 (+ 46101us) write_op.cc:301] APPLY: starting
0709 12:13:13.374517 (+   138us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:13.611131 (+236614us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:13.611232 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:13.613920 (+  2688us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:13.614175 (+   255us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:13:13.614224 (+    49us) write_op.cc:312] APPLY: finished
0709 12:13:13.614486 (+   262us) log.cc:844] Serialized 49 byte log entry
0709 12:13:13.614639 (+   153us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:13.614733 (+    94us) write_op.cc:454] Released schema lock
0709 12:13:13.614768 (+    35us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:13.620811 (+  6043us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:13.620850 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":894949,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":45710,"apply.run_cpu_time_us":9439,"apply.run_wall_time_us":246402,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20855,"num_ops":4,"prepare.queue_time_us":340009,"prepare.run_cpu_time_us":11309,"prepare.run_wall_time_us":485860,"raft.queue_time_us":128989,"raft.run_cpu_time_us":7901,"raft.run_wall_time_us":67206,"replication_time_us":267971,"spinlock_wait_cycles":4992,"thread_start_us":369243,"threads_started":4,"wal-append.queue_time_us":43538}]]}
I20260709 12:13:13.912501 18050 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:13.913158 18050 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.913583 18050 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:13.916057 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:13.917690 18160 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:13.918207 18160 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:13.918639 18160 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:13.925638 18165 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:13.926514 18162 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:13.939421 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:13.941121 18373 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.944219 18373 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.979488 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.950867 18371 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:13.980387 18371 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.950966 18357 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:13.951889 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.953727 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:13.983245 18357 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.983752 18357 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:13.984882 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.946202 18050 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.963681 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:13.962683 18104 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:13.967780 10431 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:13:13.969719 10430 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:13.971272 10578 raft_consensus.cc:1275] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:13.973237 10437 raft_consensus.cc:1275] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:13:13.974535 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.991449 18371 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.992256 18373 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:13.992858 18373 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:13.993322 17870 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:13.994596 10434 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 22: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:13.947949 10335 catalog_manager.cc:5697] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.982651 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:13.996860 10435 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:13:13.986720 10432 raft_consensus.cc:1275] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:13.996416 10335 catalog_manager.cc:5697] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:14.000588 18371 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.004236 18371 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.006951 18371 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.000586 18050 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:13.999418 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:13.998778 10335 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.999018 18373 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.013072 18291 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.030696 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:14.031587 10432 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
I20260709 12:13:14.031826 17870 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:14.035205 11177 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:14.039198 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:14.040261 10435 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 24: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:14.027628 10552 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.053578 11217 raft_consensus.cc:3060] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEARNER]: Advancing to term 15
I20260709 12:13:14.053445 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:14.058403 10432 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:14.063982 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.064780 18197 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:14.065302 18197 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.065591 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.066893 11219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:14.066313 10335 catalog_manager.cc:5697] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:14.067764 11219 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 22: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:14.068363 18281 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:14.024768 18288 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:13:14.057193 18160 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.065490 11218 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:14.065716 18197 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:14.088593 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.072626 18160 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:14.088493 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.073084 10335 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 17 to 19, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:14.110425 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:14.123309 11221 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:14.124032 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:14.124459 18291 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:14.112391 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:14.115002 17903 consensus_queue.cc:1048] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:14.119103 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.120711 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:14.121560 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 65496.7us
I20260709 12:13:14.121693 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.122445 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.125299 18356 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:14.128477 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.137987 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.142028 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.151990 18288 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:14.166653 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.173870 18030 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:14.161499 10335 catalog_manager.cc:5697] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 16, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 16 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:14.181171 10335 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 19 to 21, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 21 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:14.166826 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.174444 18130 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:14.202332 10587 raft_consensus.cc:1275] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:14.206780 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:14.207607 10584 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:14.197628 18291 raft_consensus.cc:2749] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:14.203579 10508 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 24.
I20260709 12:13:14.129829 18289 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:14.216781 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104651us
I20260709 12:13:14.218505 10479 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:14.229100 18162 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:14.229640 18162 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:14.230037 18162 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:14.186133 18357 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.255960 10552 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:14.262814 10583 raft_consensus.cc:2368] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:13:14.265702 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:13:14.268702 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 173727us
I20260709 12:13:14.289301 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.295809 17778 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:14.296427 17778 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.296846 17778 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:14.297420 18289 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:14.304514 18269 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:14.305022 18357 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:14.307404 10587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:14.308480 10587 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
W20260709 12:13:14.324504 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 59831.9us
I20260709 12:13:14.330459 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
W20260709 12:13:14.337409 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 98420.2us
I20260709 12:13:14.338771 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.352830 10584 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:14.344818 18281 consensus_queue.cc:1048] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:13:14.360010 18046 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:14.360842 18046 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.361918 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:14.363495 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 109245us
I20260709 12:13:14.364336 18285 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:14.365257 18285 consensus_queue.cc:1048] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:14.265420 10579 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:14.379452 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.379897 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.380824 10512 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 22: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:14.396667 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.398180 10403 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.398507 18089 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:14.399237 18357 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:14.408396 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.353335 18165 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:14.438567 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.444388 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:14.458621 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 253695us
I20260709 12:13:14.440506 17870 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:14.472998 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.480124 18347 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.479720 17778 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.449659 18165 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.366964 18285 consensus_queue.cc:1048] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:13:14.460383 18291 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.497741 11217 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:13:14.504281 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.485785 18165 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:14.518102 11221 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:14.561371 18046 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.510926 17870 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:14.566260 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.515278 18343 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:14.514374 18356 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:14.506369 18291 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.514052 18346 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:14.511647 18347 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.576382 17778 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:14.588630 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 340804us
W20260709 12:13:14.590484 18375 log.cc:927] Time spent T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:13:14.637835 18347 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:14.608043 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:14.627900 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:14.639649 10430 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:13:14.650357 18197 consensus_meta.cc:287] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.004s
I20260709 12:13:14.650906 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:14.651063 18197 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.652527 11216 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 21: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:14.652792 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:14.653663 11217 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 22: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:14.660676 18162 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.666283 10436 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 22: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:14.668368 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:14.669456 18356 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.671912 18356 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.672400 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.673552 10506 raft_consensus.cc:2393] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 22: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:14.674896 18162 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.676110 18291 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:14.662456 18165 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.630650 18371 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.689889 10479 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:14.701807 10435 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:14.707594 10435 raft_consensus.cc:2393] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 24: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:14.689921 17778 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.715255 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:14.716681 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:14.724875 18305 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:14.725601 18305 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:14.727200 18130 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:14.727852 10438 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:14.731081 18372 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.731640 18372 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:14.737030 18197 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.746482 18165 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:14.689447 18050 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:13:14.771611 10430 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:13:14.700381 18371 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:13:14.774047 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 504875us
I20260709 12:13:14.782374 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.748579 17778 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:14.790498 10504 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:14.804335 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.726796 18305 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:14.788146 18372 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.847362 10404 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:14.848532 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.849274 10512 raft_consensus.cc:3060] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:14.850026 18371 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:14.861701 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:14.863003 10430 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:14.864326 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.877624 10478 leader_election.cc:400] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
W20260709 12:13:14.878484 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50248.1us
I20260709 12:13:14.880683 10513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.893304 17778 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:14.896710 10509 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 24.
I20260709 12:13:14.899591 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:14.906692 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:14.909013 10581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.935156 10581 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:13:14.934881 10512 raft_consensus.cc:2468] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:13:14.940945 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.943701 11217 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEARNER]: Committing config change with OpId 13.9: config changed from index -1 to 9, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:13:14.936797 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 89572.1us
I20260709 12:13:14.954355 10586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:14.956158 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:14.957124 10505 raft_consensus.cc:2410] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:13:14.958799 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.959874 10586 raft_consensus.cc:2368] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:13:14.965916 18333 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:14.966074 10553 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:14.967386 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:14.968453 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:14.968636 18281 raft_consensus.cc:2804] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:14.969363 18281 raft_consensus.cc:697] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:15.057197 10583 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 23. Current term is 24. Ops: []
W20260709 12:13:15.060058 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 75759.1us
I20260709 12:13:15.081832 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:15.097790 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:15.099056 10580 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 21: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:15.100312 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:14.983882 18356 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:15.104904 10504 raft_consensus.cc:2393] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 21: Already voted for candidate 99d42e7288f145f1bd5af212fd062995 in this term.
I20260709 12:13:15.106560 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:15.107195 18372 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:15.108960 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:15.126200 11177 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:14.970619 18281 consensus_queue.cc:237] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:15.128989 18385 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:13:15.213049 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 131783us
I20260709 12:13:15.214418 18305 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:13:15.235499 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 280604us
I20260709 12:13:15.250590 10508 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41084 (request call id 3983) took 1023 ms. Trace:
W20260709 12:13:15.254344 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 197436us
I20260709 12:13:15.263897 10552 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.267694 10433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:15.268946 10433 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:13:15.261174 10508 rpcz_store.cc:276] 0709 12:13:14.227294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.245137 (+ 17843us) service_pool.cc:224] Handling call
0709 12:13:14.269242 (+ 24105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:14.589936 (+320694us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18003af180. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f23f3 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:13:14.702636 (+112700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:14.702687 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:14.775182 (+ 72495us) log.cc:844] Serialized 26 byte log entry
0709 12:13:15.233337 (+458155us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:15.233486 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.234797 (+  1311us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.250136 (+ 15339us) raft_consensus.cc:1685] finished
0709 12:13:15.250160 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.250449 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40142,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":7921536,"thread_start_us":742029,"threads_started":2,"wal-append.queue_time_us":448732}
I20260709 12:13:15.273324 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:15.273939 18332 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:15.274582 18332 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:15.270180 18305 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:15.276995 18332 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:15.279933 18305 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:15.289126 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:15.295178 18332 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:15.295951 18332 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:15.298744 18332 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:15.299703 18388 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:15.300398 18388 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:15.301457 18388 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:15.309124 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:15.312150 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:15.314018 10505 raft_consensus.cc:2410] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:13:15.313781 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.315721 18393 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:15.316015 10478 leader_election.cc:400] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:13:15.316366 10478 leader_election.cc:403] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:13:15.317591 18130 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727) with higher term. Message: Invalid argument: T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 17. Current term is 18.
I20260709 12:13:15.321899 18347 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:15.322697 18347 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:15.325711 18347 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:15.330878 18393 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:15.331496 18332 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:15.332367 18393 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:15.332886 18393 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:15.334026 18393 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:15.336434 10586 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:13:15.337201 10579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:15.343433 11178 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.332209 18332 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:15.344909 18347 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:15.345515 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.346369 18347 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:15.346923 18332 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:15.347337 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.350592 18347 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:15.348527 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:15.347787 18393 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:15.363165 18241 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:15.363945 18241 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:15.368420 18241 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:15.379863 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 238893us
W20260709 12:13:15.380421 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45639.4us
I20260709 12:13:15.385453 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
W20260709 12:13:15.387403 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 46084.6us
I20260709 12:13:15.389912 10553 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.397348 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:15.406579 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:15.401932 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:15.406559 10437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:15.416477 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:15.439692 17870 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:15.441639 18309 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:15.444036 11220 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:13:15.422791 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:15.448339 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 56132.4us
I20260709 12:13:15.452626 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:15.453490 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:15.455304 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:15.451965 10478 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:15.461766 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 65377us
I20260709 12:13:15.461151 18333 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:15.447724 18288 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:15.463085 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.469146 11178 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.472344 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:15.468844 10508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:15.456002 18332 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:15.489470 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:15.505498 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:15.507517 18281 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:15.512717 18030 raft_consensus.cc:1064] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:13:15.476593 18073 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:13:15.513855 18429 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:15.514276 10512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:15.515260 10512 raft_consensus.cc:606] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:15.523180 18394 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:15.527140 18391 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:15.528020 18410 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:15.528630 18409 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:15.528671 18410 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:15.537768 18410 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:15.539495 11216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:15.500901 17778 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:15.547211 10431 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31117e61be004c52a161168a559023fd"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:15.547878 10431 raft_consensus.cc:606] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:15.548327 10431 raft_consensus.cc:614] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:15.558027 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:15.560431 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:15.562306 18373 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:15.573078 10582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "448b86aa128243e48bed5f471ada874f"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:13:15.573781 10582 raft_consensus.cc:606] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:15.558753 18030 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:15.583123 10431 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:15.589476 10431 raft_consensus.cc:606] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:15.651746 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 165894us
W20260709 12:13:15.655792 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 165407us
I20260709 12:13:15.709614 10431 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:13:15.711158 10429 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:15.712976 10429 raft_consensus.cc:606] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:15.713320 10429 raft_consensus.cc:614] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:15.726845  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:15.745929 18413 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:13:15.791219 10582 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:15.795840  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:15.803433 11216 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:15.812451 18430 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:15.826474 10508 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:15.829021  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:15.832350 18432 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:15.828222 10431 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:15.861883 18388 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:15.865621 10508 raft_consensus.cc:1275] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:13:15.910419 10328 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:15.912712 10328 catalog_manager.cc:5697] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 12 to 13, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 13 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:15.920003 18388 consensus_queue.cc:1048] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:15.921159 18130 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:15.960502 10583 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 9 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:15.976979 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 173) took 7351 ms. Trace:
I20260709 12:13:15.955727 10328 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 20, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), now has a pending config: VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67), VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } } pending_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:15.983253 10328 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 9, term changed from 13 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to <none>, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 15 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:15.986119 18343 raft_consensus.cc:993] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:15.985214 10328 catalog_manager.cc:5697] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:15.987766 10328 catalog_manager.cc:5697] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:15.986742 18343 raft_consensus.cc:1081] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:15.944183 18432 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 9 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:16.005654 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 46505.3us
I20260709 12:13:16.007735 10328 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:15.973834 11220 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 9 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:16.015138 18436 log.cc:927] Time spent T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:13:16.019330 18073 raft_consensus.cc:993] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:16.020061 18073 raft_consensus.cc:1081] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:16.023849 10328 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 18, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:16.026496 10328 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 20 to 23, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 23 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:16.029322 10328 catalog_manager.cc:5697] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:15.977502 10335 rpcz_store.cc:276] 0709 12:13:08.641812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.647964 (+  6152us) service_pool.cc:224] Handling call
0709 12:13:15.976901 (+7328937us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:14.896812 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:14.897837 (+  1025us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:14.897909 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:13:14.897939 (+    30us) tablet.cc:662] Decoding operations
0709 12:13:14.968549 (+ 70610us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:14.968686 (+   137us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:14.968722 (+    36us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:13:14.969566 (+   844us) tablet.cc:701] Row locks acquired
0709 12:13:14.969621 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:13:14.969844 (+   223us) write_op.cc:270] Start()
0709 12:13:14.970011 (+   167us) write_op.cc:276] Timestamp: P: 1783599194969765 usec, L: 0
0709 12:13:14.970052 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:13:15.123208 (+153156us) log.cc:844] Serialized 3170 byte log entry
0709 12:13:15.373229 (+250021us) op_driver.cc:464] REPLICATION: finished
0709 12:13:15.507790 (+134561us) write_op.cc:301] APPLY: starting
0709 12:13:15.507931 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:15.552022 (+ 44091us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b5400366854. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c92a48571 00007f2c92adc32a 00007f2c92a61f14 00007f2c92a61ec3 00007f2c92a61e38 00007f2c92a61df3 00007f2c92a610dc 00007f2c92a8ae16 00007f2c92a72cbb 00007f2c92a5d39d 00007f2c92a60d75
0709 12:13:15.578381 (+ 26359us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:15.578429 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:15.672241 (+ 93812us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:15.672597 (+   356us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:13:15.672671 (+    74us) write_op.cc:312] APPLY: finished
0709 12:13:15.673838 (+  1167us) log.cc:844] Serialized 81 byte log entry
0709 12:13:15.682330 (+  8492us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:15.682493 (+   163us) write_op.cc:454] Released schema lock
0709 12:13:15.682590 (+    97us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:15.725084 (+ 42494us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:15.727121 (+  2037us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":67629,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":134162,"apply.run_cpu_time_us":17478,"apply.run_wall_time_us":217288,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37583,"num_ops":8,"prepare.queue_time_us":497033,"prepare.run_cpu_time_us":46076,"prepare.run_wall_time_us":542348,"raft.queue_time_us":1456,"raft.run_cpu_time_us":54006,"raft.run_wall_time_us":187102,"replication_time_us":402931,"spinlock_wait_cycles":17997184,"thread_start_us":639770,"threads_started":4,"wal-append.queue_time_us":228554}]]}
I20260709 12:13:16.035183 11221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57904
I20260709 12:13:16.033875 10328 catalog_manager.cc:5697] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 20, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:16.043448 11221 raft_consensus.cc:493] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:16.043912 11221 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:16.047911 10430 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:16.056231 10328 catalog_manager.cc:5697] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:16.069061 10328 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:16.091737 10328 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:16.095425 10328 catalog_manager.cc:5697] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 20, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:16.097091 10328 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 20 to 21, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 21 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:16.096097 11212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57890
I20260709 12:13:16.103009 11212 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:16.102559 10328 catalog_manager.cc:5697] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 19 to 21, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 21 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:16.103430 11212 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:16.090626 11221 raft_consensus.cc:515] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.068833 18413 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 12:13:16.156838 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 100080us
I20260709 12:13:16.143818 18413 consensus_queue.cc:1243] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [LEADER]: Peer 99d42e7288f145f1bd5af212fd062995 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:13:16.168018 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 98715.6us
I20260709 12:13:16.236080 11212 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:16.275977 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 101687us
W20260709 12:13:16.349139 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 45410.4us
I20260709 12:13:16.355629 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:16.356467 10511 raft_consensus.cc:3055] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:16.356828 10511 raft_consensus.cc:740] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:16.357678 10511 consensus_queue.cc:260] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:16.451503 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 169500us
W20260709 12:13:16.461169 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.462052 18389 raft_consensus.cc:993] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:13:16.463704 18288 raft_consensus.cc:1081] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
W20260709 12:13:16.491855 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 203502us
I20260709 12:13:16.541271 10431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:16.542069 10431 raft_consensus.cc:3055] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:16.542452 10431 raft_consensus.cc:740] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:16.543375 10431 consensus_queue.cc:260] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.544570 10431 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:16.575443 18456 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:16.592123 18456 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.598354 10511 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:16.600701 18456 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:16.606482 10431 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
W20260709 12:13:16.607297 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.608527 18458 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:16.609158 18458 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.616546 10511 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:16.634203 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:16.635049 10510 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:16.638705 11212 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:16.644764 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:16.645597 10578 raft_consensus.cc:3060] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:16.647231 10504 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "448b86aa128243e48bed5f471ada874f"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:13:16.655326 11221 leader_election.cc:290] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:16.647322 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:16.662999 18458 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:16.675477 10583 raft_consensus.cc:3060] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:16.641474 11178 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
W20260709 12:13:16.684567 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.686288 18452 raft_consensus.cc:2804] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:13:16.687011 18452 raft_consensus.cc:697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:16.688014 18452 consensus_queue.cc:237] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.697562 10511 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33776
I20260709 12:13:16.698546 10511 raft_consensus.cc:606] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Received request to transfer leadership
I20260709 12:13:16.699299 10511 raft_consensus.cc:614] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:16.717067 10583 raft_consensus.cc:2468] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:16.714174 10434 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 9 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:16.718906 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:16.724339 11217 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:16.729053 10504 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:16.729632 10504 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:16.731684 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 48390.4us
I20260709 12:13:16.734586 10578 raft_consensus.cc:2468] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 16.
I20260709 12:13:16.739545 10432 pending_rounds.cc:85] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 12
I20260709 12:13:16.735186 11177 leader_election.cc:304] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:13:16.747820 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 106940us
I20260709 12:13:16.746737 10403 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:16.740033 10432 pending_rounds.cc:107] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:13:16.768697 10432 raft_consensus.cc:2889] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:16.776540 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:16.774144 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:16.780092 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 133272us
I20260709 12:13:16.777952 10582 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:16.766850 18441 raft_consensus.cc:2804] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:16.771152 18449 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:16.787822 18449 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.788316 18449 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:16.792263 10511 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
I20260709 12:13:16.793664 18441 raft_consensus.cc:697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:16.794929 18441 consensus_queue.cc:237] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.781795 10504 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:16.810263 10384 auto_rebalancer.cc:893] Removing replace marker failed: Replica 96a7b6a7b75246d381f5d4a71ea716d9 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } } pending_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:16.810541 10504 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 25 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:16.812217 10384 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:13:16.812659 10384 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:13:16.830684 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:16.831732 11212 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 21.
W20260709 12:13:16.861934 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49869.2us
I20260709 12:13:16.865976 10404 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:16.869004 18373 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:16.869843 18373 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.870206 18373 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:16.879007 18444 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:16.879722 18444 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:16.884663 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:16.885486 10504 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 24.
I20260709 12:13:16.887246 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:16.887737 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:16.888648 10585 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 24: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:16.892171 18333 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:16.892738 18333 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:16.893250 18333 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:16.900192 10336 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:16.907171 18449 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.913496 18449 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:16.915316 18444 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:16.915331 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:16.916443 11217 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:16.917955 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48697.3us
I20260709 12:13:16.926513 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:16.927222 10511 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:16.929641 11217 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 23.
I20260709 12:13:16.931079 10404 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:16.931985 18444 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:16.932502 18444 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 23 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:16.933504 18444 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.936122 10511 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 23.
W20260709 12:13:16.942047 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 74194.2us
I20260709 12:13:16.946833 10336 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: leader changed from <none> to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), now has a pending config: VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67), VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:16.953320 10336 catalog_manager.cc:5697] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 20, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:16.955498 10328 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 294) took 2985 ms. Trace:
I20260709 12:13:16.955672 10336 catalog_manager.cc:5697] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:16.958351 10336 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: config changed from index 12 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:16.956002 10328 rpcz_store.cc:276] 0709 12:13:13.969860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.970411 (+   551us) service_pool.cc:224] Handling call
0709 12:13:16.955392 (+2984981us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:16.260031 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:16.260682 (+   651us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:16.260720 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:13:16.261100 (+   380us) tablet.cc:662] Decoding operations
0709 12:13:16.263360 (+  2260us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:16.263460 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:16.263490 (+    30us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:13:16.355671 (+ 92181us) tablet.cc:701] Row locks acquired
0709 12:13:16.355697 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:13:16.355900 (+   203us) write_op.cc:270] Start()
0709 12:13:16.356056 (+   156us) write_op.cc:276] Timestamp: P: 1783599196355842 usec, L: 0
0709 12:13:16.356078 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:13:16.357061 (+   983us) log.cc:844] Serialized 6950 byte log entry
0709 12:13:16.543899 (+186838us) op_driver.cc:464] REPLICATION: finished
0709 12:13:16.629002 (+ 85103us) write_op.cc:301] APPLY: starting
0709 12:13:16.629109 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:16.649603 (+ 20494us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:16.649760 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:16.662847 (+ 13087us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:16.663267 (+   420us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 12:13:16.663338 (+    71us) write_op.cc:312] APPLY: finished
0709 12:13:16.663985 (+   647us) log.cc:844] Serialized 156 byte log entry
0709 12:13:16.664248 (+   263us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:16.664509 (+   261us) write_op.cc:454] Released schema lock
0709 12:13:16.664577 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:16.668628 (+  4051us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:16.668680 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36262,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":84785,"apply.run_cpu_time_us":24849,"apply.run_wall_time_us":39567,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75221,"num_ops":17,"prepare.queue_time_us":95127,"prepare.run_cpu_time_us":6535,"prepare.run_wall_time_us":97511,"replication_time_us":187587,"spinlock_wait_cycles":54016,"thread_start_us":109634,"threads_started":2}]]}
I20260709 12:13:16.985280 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:16.989324 18373 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:16.997128 10336 catalog_manager.cc:5697] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:17.008338 18373 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:17.012426 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:17.013269 11217 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:17.022078 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:17.023028 10585 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:17.028016 10583 raft_consensus.cc:1240] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Rejecting Update request from peer 96a7b6a7b75246d381f5d4a71ea716d9 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:17.028527 10336 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:17.037094 10336 catalog_manager.cc:5697] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:17.036458 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:17.057753 10505 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:17.039347 18333 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:17.070504 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:17.074096 10583 raft_consensus.cc:3055] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:17.074530 10583 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:17.075366 10583 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:17.067122 10476 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.076402 10583 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:17.046736 11217 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:17.078294 10505 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:17.091667 10506 raft_consensus.cc:1240] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Rejecting Update request from peer 33d77e3063684586a737bccf358786d8 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:17.093672 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:17.070969 10336 catalog_manager.cc:5697] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 22 to 23, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 23 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:17.098851 10403 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:17.101497 10585 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 22.
I20260709 12:13:17.092583 18333 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:17.102358 10336 catalog_manager.cc:5697] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 18, leader changed from 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 18 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:17.109764 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:17.102430 10583 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 25.
I20260709 12:13:17.112452 10580 raft_consensus.cc:2393] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 99d42e7288f145f1bd5af212fd062995 in current term 24: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:17.135521  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:17.139124 17903 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.139915 17903 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:17.115448 18449 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:17.146044 18449 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:17.147176 18449 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:17.163626 10506 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 24.
I20260709 12:13:17.179337 10478 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:17.182215 18447 raft_consensus.cc:2804] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:17.182835 18447 raft_consensus.cc:697] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:17.183900 18447 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:17.195253 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8
I20260709 12:13:17.195932 10436 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:13:17.201864 18448 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:17.217957 17903 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:17.210502 18448 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 24 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:17.219719 18448 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:17.240669 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:17.246003 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:17.250780 10506 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 24.
I20260709 12:13:17.248986 18441 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:13:17.257696 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:17.259240 18414 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:17.259708 18414 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.260076 18414 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:17.271122 10583 raft_consensus.cc:1275] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:13:17.273303 18415 consensus_queue.cc:1048] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:17.292596 10505 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:13:17.308640 18415 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:13:17.350082 10580 raft_consensus.cc:1275] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:13:17.355759 11221 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:17.358100 18415 consensus_queue.cc:1048] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:17.366358 10505 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:17.369213 18373 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:17.412640 18373 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:17.470530 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 162477us
I20260709 12:13:17.454885 18414 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:17.476364 18433 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:17.530942 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06d2724d97aa43298ebaf9e746580fd0"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:17.532030 10505 raft_consensus.cc:606] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:17.532543 10505 raft_consensus.cc:614] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:17.548278 18414 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 25 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:17.556672 18470 log.cc:927] Time spent T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.153s	user 0.001s	sys 0.002s
I20260709 12:13:17.513166 18433 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:17.564702 18433 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:17.567696 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:17.568759 10512 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:17.568897 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995" is_pre_election: true
I20260709 12:13:17.587934 10506 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:17.602895 18373 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:17.607322 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:17.627560 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:17.628644 10505 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:17.695384 10512 raft_consensus.cc:2468] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 25.
I20260709 12:13:17.699033 10552 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:17.704037 10432 raft_consensus.cc:3060] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:17.705168 10505 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:17.708983 18388 raft_consensus.cc:2804] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:13:17.730654 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 236362us
I20260709 12:13:17.734366 10512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:17.743409 18388 raft_consensus.cc:697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 25 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:17.745121 18388 consensus_queue.cc:237] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:17.797251 10432 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:17.891693 10432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
mode: GRACEFUL
new_leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:49374
I20260709 12:13:17.892520 10432 raft_consensus.cc:606] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Received request to transfer leadership to TS 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:17.904906 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 87903.8us
W20260709 12:13:17.914960 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 139493us
I20260709 12:13:17.961869 11177 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:17.956756 18467 log.cc:927] Time spent T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:13:18.003481 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 79571.9us
W20260709 12:13:18.005113 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 47787.6us
I20260709 12:13:18.037580 18447 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:18.060694 11212 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:18.054455 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:18.072268 10505 raft_consensus.cc:606] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:18.016336 18496 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:18.098817 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:18.122534 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 102173us
I20260709 12:13:18.093978 18346 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:13:18.131233 10585 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:13:18.097512 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 109331us
I20260709 12:13:18.178054 10582 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:18.215798 10585 raft_consensus.cc:1240] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 25 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:18.216692 18387 raft_consensus.cc:993] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:18.150175 18444 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:13:18.217330 18387 raft_consensus.cc:1081] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
W20260709 12:13:18.241446 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 57493.5us
I20260709 12:13:18.255530 18491 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:18.281987 10582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:13:18.282855 10582 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:18.283473 10582 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:18.274683 18510 raft_consensus.cc:3055] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:18.283994 18510 raft_consensus.cc:740] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:18.285223 18510 consensus_queue.cc:260] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:18.291554 18444 consensus_queue.cc:1059] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:18.292307 18444 consensus_queue.cc:1243] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Peer 33d77e3063684586a737bccf358786d8 log is divergent from this leader: its last log entry 25.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:13:18.293015 18333 raft_consensus.cc:993] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: : Instructing follower 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:18.293587 18333 raft_consensus.cc:1081] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Signalling peer 52bae2d2caf94d97b1049ecd4f82a2df to start an election
I20260709 12:13:18.299386 11217 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9"
dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
 from {username='slave'} at 127.0.0.1:57890
I20260709 12:13:18.299940 11217 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:18.300272 11217 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.302019 10510 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:18.303810 18511 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:18.310123 17903 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:18.322166 10582 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:18.363094 10505 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:18.364285 10505 raft_consensus.cc:606] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:18.392619 10582 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:18.417375 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:18.334940 18510 raft_consensus.cc:3060] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:18.418215 10511 raft_consensus.cc:3055] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:13:18.418916 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 64488.9us
I20260709 12:13:18.426810  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:18.427022 10511 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:18.432089  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:18.426810 11217 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:18.448269 10511 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:18.463897 11217 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:18.475965 10511 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:13:18.490796 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.500190 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:18.501035 10504 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.511096 10511 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:18.513280 10552 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:13:18.517396 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 90014.2us
I20260709 12:13:18.517946 10504 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
I20260709 12:13:18.530673 10430 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:18.533959 10430 raft_consensus.cc:3060] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:18.533188 11177 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:18.536499 18505 raft_consensus.cc:993] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:18.537107 18505 raft_consensus.cc:1081] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:18.560465 10430 raft_consensus.cc:2468] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:18.565984 18489 raft_consensus.cc:2804] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:18.566754 18489 raft_consensus.cc:697] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:18.567799 18489 consensus_queue.cc:237] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:18.525470 18516 raft_consensus.cc:2804] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:13:18.573282 18516 raft_consensus.cc:697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:18.574373 18516 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:18.581563 18521 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:13:18.621666 10511 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:18.670533 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 145045us
I20260709 12:13:18.697624 10583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:13:18.698326 10583 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:18.698704 10583 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:18.710799 10583 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:18.711599 10431 raft_consensus.cc:1275] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:13:18.712083 10431 pending_rounds.cc:85] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting all ops after (but not including) 14
I20260709 12:13:18.712323 10431 pending_rounds.cc:107] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:13:18.712558 10431 raft_consensus.cc:2889] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:18.714902 10436 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:18.715651 10436 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:18.731841 18481 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:18.732692 18481 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:18.748225 10583 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:18.758311 10429 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:18.759330 10429 raft_consensus.cc:3055] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:18.759768 10429 raft_consensus.cc:740] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:18.764086 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:18.764851 10504 raft_consensus.cc:3055] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:18.765246 10504 raft_consensus.cc:740] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:18.782078 10504 consensus_queue.cc:260] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:18.795430 10504 raft_consensus.cc:3060] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:18.804612 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:18.791276 18507 raft_consensus.cc:993] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
W20260709 12:13:18.805683 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51395.6us
I20260709 12:13:18.798655 10585 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:18.819942 10436 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:18.783982 10429 consensus_queue.cc:260] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:18.829362 10429 raft_consensus.cc:3060] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.824851 11212 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:18.834285 10511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:18.802527 18481 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:18.829568 18444 consensus_queue.cc:1175] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" responder_term: 22 status { last_received { term: 21 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:18.846342 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:18.830130 18285 raft_consensus.cc:1081] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:18.855382 11217 raft_consensus.cc:1240] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Rejecting Update request from peer 99d42e7288f145f1bd5af212fd062995 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:18.836057 10505 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:18.859668 18452 raft_consensus.cc:2749] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:18.899115 18510 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Committing config change with OpId 20.13: config changed from index 11 to 13, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:18.433570  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:18.858706 10504 raft_consensus.cc:2468] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
W20260709 12:13:18.921120 10402 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.935736 10429 raft_consensus.cc:2468] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 22.
W20260709 12:13:18.955724 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 169133us
W20260709 12:13:18.974545 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 180859us
I20260709 12:13:18.977689 10553 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:18.984776 10582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:13:18.985791 10582 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:18.986302 10582 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:19.019634 10582 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:19.024838 10434 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:19.025547 10434 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:19.026818 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:19.025200 10582 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:19.027480 10505 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:13:19.027946 10505 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:19.028807 10505 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:19.030093 10505 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:19.005788 18527 raft_consensus.cc:2804] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:18.998607 17903 consensus_queue.cc:1048] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:13:19.049732 10429 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:13:19.056705 10505 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:13:19.064543 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:19.056483 18464 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:13:19.065770 18516 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:13:19.066402 18516 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:19.067435 18516 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:19.049000 18527 raft_consensus.cc:697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:19.096228 18527 consensus_queue.cc:237] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:19.112355 10429 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:19.139930 10336 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46470 (request call id 303) took 3355 ms. Trace:
I20260709 12:13:19.141777 10504 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:19.158054 18452 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:19.062220 10434 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:13:19.165302 10332 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 21 to 23, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 23 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:19.192836 10505 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:13:19.141685 10336 rpcz_store.cc:276] 0709 12:13:15.784159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.784445 (+   286us) service_pool.cc:224] Handling call
0709 12:13:19.139838 (+3355393us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:17.224799 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:17.225415 (+   616us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:17.225456 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:13:17.225474 (+    18us) tablet.cc:662] Decoding operations
0709 12:13:17.226578 (+  1104us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:17.226679 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:17.226707 (+    28us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:13:17.227549 (+   842us) tablet.cc:701] Row locks acquired
0709 12:13:17.227570 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:13:17.235349 (+  7779us) write_op.cc:270] Start()
0709 12:13:17.261099 (+ 25750us) write_op.cc:276] Timestamp: P: 1783599197227694 usec, L: 0
0709 12:13:17.265673 (+  4574us) op_driver.cc:348] REPLICATION: starting
0709 12:13:17.266510 (+   837us) log.cc:844] Serialized 4304 byte log entry
0709 12:13:17.532258 (+265748us) op_driver.cc:464] REPLICATION: finished
0709 12:13:17.762535 (+230277us) write_op.cc:301] APPLY: starting
0709 12:13:17.762646 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:17.809441 (+ 46795us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:17.809582 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:18.162495 (+352913us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:18.162725 (+   230us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:13:18.162758 (+    33us) write_op.cc:312] APPLY: finished
0709 12:13:18.163150 (+   392us) log.cc:844] Serialized 97 byte log entry
0709 12:13:18.177129 (+ 13979us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:18.198629 (+ 21500us) write_op.cc:454] Released schema lock
0709 12:13:18.218614 (+ 19985us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:18.219179 (+   565us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:18.219219 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":65877,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":225134,"apply.run_cpu_time_us":19655,"apply.run_wall_time_us":456625,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":90860,"prepare.run_cpu_time_us":4578,"prepare.run_wall_time_us":42146,"replication_time_us":250334,"thread_start_us":248935,"threads_started":2}]]}
I20260709 12:13:19.201414 18531 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:13:19.259759 10582 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:13:19.267266 18479 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:19.284014 10505 raft_consensus.cc:1275] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:19.278352 11212 raft_consensus.cc:1275] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:13:19.310590 18532 consensus_queue.cc:1048] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:19.310925 18503 consensus_queue.cc:1048] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:13:19.517443 10506 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:19.521166 18516 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:13:19.534896 18553 log.cc:927] Time spent T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:13:19.570432 18512 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:19.585249 10582 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:19.608147 11221 raft_consensus.cc:2955] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:19.657541 10432 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:13:19.661516 18508 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:19.784473 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 945dd7b821a5493f8a9d6ca251795da9 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:19.795794 10504 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:13:19.797695 18516 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:19.820036 10432 raft_consensus.cc:1275] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:13:19.832778 18508 consensus_queue.cc:1048] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:19.969506 10328 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 9 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67), VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:19.971788 10328 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:19.974143 10328 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 23 to 24, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 24 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:19.976418 10328 catalog_manager.cc:5697] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 13 to 14, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 14 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.011612 10328 catalog_manager.cc:5697] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 16 to 18, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 18 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:20.068410 10332 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46484 (request call id 314) took 5866 ms. Trace:
W20260709 12:13:20.072232 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60269.3us
I20260709 12:13:20.069121 10332 rpcz_store.cc:276] 0709 12:13:14.206051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.208115 (+  2064us) service_pool.cc:224] Handling call
0709 12:13:20.068312 (+5860197us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:19.272964 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:19.273820 (+   856us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:19.273871 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:13:19.273893 (+    22us) tablet.cc:662] Decoding operations
0709 12:13:19.274280 (+   387us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:19.274365 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:19.274404 (+    39us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:19.274597 (+   193us) tablet.cc:701] Row locks acquired
0709 12:13:19.274615 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:13:19.274840 (+   225us) write_op.cc:270] Start()
0709 12:13:19.274965 (+   125us) write_op.cc:276] Timestamp: P: 1783599199274784 usec, L: 0
0709 12:13:19.301203 (+ 26238us) op_driver.cc:348] REPLICATION: starting
0709 12:13:19.334542 (+ 33339us) log.cc:844] Serialized 535 byte log entry
0709 12:13:19.574170 (+239628us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b1800394780. stack: 00007f2c8f9c0343 00007f2c8f9c0240 00007f2c8f34f1d4 00007f2c9908d304 00007f2c9908d299 00007f2c988baab1 00007f2c988c7281 00007f2c8f9f2522 00007f2c8f9f47c1 00007f2c8f9f4779 00007f2c8f9f4709 00007f2c8f9f46d1 00007f2c8f9f39cd 0000000000388634 0000000000388479 00007f2c8f9d583a
0709 12:13:19.711292 (+137122us) op_driver.cc:464] REPLICATION: finished
0709 12:13:19.900412 (+189120us) write_op.cc:301] APPLY: starting
0709 12:13:19.900515 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:19.901836 (+  1321us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:19.901861 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:19.913281 (+ 11420us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:19.913504 (+   223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:13:19.913560 (+    56us) write_op.cc:312] APPLY: finished
0709 12:13:19.913794 (+   234us) log.cc:844] Serialized 25 byte log entry
0709 12:13:19.914683 (+   889us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:19.914780 (+    97us) write_op.cc:454] Released schema lock
0709 12:13:19.914829 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:19.916174 (+  1345us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:19.916221 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6794,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":168285,"apply.run_cpu_time_us":3676,"apply.run_wall_time_us":15687,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":57972,"prepare.run_cpu_time_us":23826,"prepare.run_wall_time_us":174580,"raft.queue_time_us":160522,"raft.run_cpu_time_us":46493,"raft.run_wall_time_us":92351,"replication_time_us":402145,"spinlock_wait_cycles":5249280,"thread_start_us":252398,"threads_started":4,"wal-append.queue_time_us":116637}]]}
W20260709 12:13:20.167507 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 56844.3us
I20260709 12:13:20.244179 10328 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46462 (request call id 295) took 2994 ms. Trace:
I20260709 12:13:20.244724 10328 rpcz_store.cc:276] 0709 12:13:17.249745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.250556 (+   811us) service_pool.cc:224] Handling call
0709 12:13:20.244081 (+2993525us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:20.195409 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:20.196043 (+   634us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:20.196086 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:13:20.196106 (+    20us) tablet.cc:662] Decoding operations
0709 12:13:20.196808 (+   702us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:20.196898 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:20.196927 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:13:20.210053 (+ 13126us) tablet.cc:701] Row locks acquired
0709 12:13:20.210076 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:13:20.210258 (+   182us) write_op.cc:270] Start()
0709 12:13:20.210401 (+   143us) write_op.cc:276] Timestamp: P: 1783599200210200 usec, L: 0
0709 12:13:20.210427 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:13:20.211192 (+   765us) log.cc:844] Serialized 2110 byte log entry
0709 12:13:20.214164 (+  2972us) op_driver.cc:464] REPLICATION: finished
0709 12:13:20.215497 (+  1333us) write_op.cc:301] APPLY: starting
0709 12:13:20.215606 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:20.218927 (+  3321us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:20.221144 (+  2217us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:20.224607 (+  3463us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:20.224865 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:13:20.224920 (+    55us) write_op.cc:312] APPLY: finished
0709 12:13:20.225219 (+   299us) log.cc:844] Serialized 57 byte log entry
0709 12:13:20.227202 (+  1983us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:20.227346 (+   144us) write_op.cc:454] Released schema lock
0709 12:13:20.227404 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:20.232365 (+  4961us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:20.232659 (+   294us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":78610,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":1053,"apply.run_cpu_time_us":8584,"apply.run_wall_time_us":15834,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20910,"num_ops":5,"prepare.queue_time_us":105990,"prepare.run_cpu_time_us":3854,"prepare.run_wall_time_us":16164,"replication_time_us":3580,"thread_start_us":66150,"threads_started":1}]]}
I20260709 12:13:20.249879 10335 catalog_manager.cc:5697] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 20 to 21, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 21 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.261382 10335 catalog_manager.cc:5697] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 15 to 16, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 16 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.306690 10332 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 11 to 13, term changed from 20 to 22, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65), VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66), VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67), VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 22 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.309527 10332 catalog_manager.cc:5697] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 19, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 19 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.302810 10335 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45398 (request call id 174) took 3374 ms. Trace:
I20260709 12:13:20.312983 10335 rpcz_store.cc:276] 0709 12:13:16.928872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:16.929049 (+   177us) service_pool.cc:224] Handling call
0709 12:13:20.302714 (+3373665us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:20.266580 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:20.267306 (+   726us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:20.267348 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:13:20.267365 (+    17us) tablet.cc:662] Decoding operations
0709 12:13:20.267839 (+   474us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:20.267908 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:20.267940 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:13:20.268326 (+   386us) tablet.cc:701] Row locks acquired
0709 12:13:20.268350 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:13:20.268516 (+   166us) write_op.cc:270] Start()
0709 12:13:20.269217 (+   701us) write_op.cc:276] Timestamp: P: 1783599200268462 usec, L: 0
0709 12:13:20.269245 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:13:20.271501 (+  2256us) log.cc:844] Serialized 907 byte log entry
0709 12:13:20.273880 (+  2379us) op_driver.cc:464] REPLICATION: finished
0709 12:13:20.274476 (+   596us) write_op.cc:301] APPLY: starting
0709 12:13:20.274574 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:20.276062 (+  1488us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:20.276418 (+   356us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:20.277847 (+  1429us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:20.278039 (+   192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:13:20.278087 (+    48us) write_op.cc:312] APPLY: finished
0709 12:13:20.278676 (+   589us) log.cc:844] Serialized 33 byte log entry
0709 12:13:20.287181 (+  8505us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:20.287295 (+   114us) write_op.cc:454] Released schema lock
0709 12:13:20.287345 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:20.291702 (+  4357us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:20.291740 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16680,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":355,"apply.run_cpu_time_us":4434,"apply.run_wall_time_us":17152,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8364,"num_ops":2,"prepare.queue_time_us":139,"prepare.run_cpu_time_us":3385,"prepare.run_wall_time_us":5271,"replication_time_us":4452}]]}
I20260709 12:13:20.318281 10332 catalog_manager.cc:5697] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 23 to 25, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 25 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.322261 10332 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 24 to 25, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 25 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } }
I20260709 12:13:20.323782 10332 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 15 to 16, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.327850 10332 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 14 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 15 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.329943 10332 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 13 to 14, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.390579 18444 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:20.391330 18444 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:20.398083 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:20.398727 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:20.402627 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:20.402132 10336 catalog_manager.cc:5697] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 21 to 22, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 22 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.396813 18444 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:20.406756 10404 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:20.439420 18444 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:20.445075 10421 tablet_service.cc:1558] Processing DeleteTablet for tablet 945dd7b821a5493f8a9d6ca251795da9 with delete_type TABLET_DATA_TOMBSTONED (TS 99d42e7288f145f1bd5af212fd062995 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:51568
I20260709 12:13:20.497148 10506 consensus_queue.cc:237] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:20.535914 10429 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:13:20.538084 11212 raft_consensus.cc:1275] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:13:20.542548 18532 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:20.546137 18447 consensus_queue.cc:1048] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:20.546955 18565 tablet_replica.cc:333] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:20.548346 18565 raft_consensus.cc:2243] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:20.549362 18565 raft_consensus.cc:2272] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:20.584110 10434 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:20.600456 18073 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.602728 10429 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.604763 18565 ts_tablet_manager.cc:1916] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:20.623273  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:20.663985 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26d3d17d58be40c8974c348ca146a951 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:20.682137 10336 catalog_manager.cc:5697] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New cstate: current_term: 15 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.684657 11212 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:20.695734 11217 raft_consensus.cc:2955] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 33d77e3063684586a737bccf358786d8 (127.9.11.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.699978 10582 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:20.709798 18564 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:20.716660 18564 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:20.718715 10569 tablet_service.cc:1558] Processing DeleteTablet for tablet 26d3d17d58be40c8974c348ca146a951 with delete_type TABLET_DATA_TOMBSTONED (TS 33d77e3063684586a737bccf358786d8 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:55454
I20260709 12:13:20.769500 10580 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:13:20.770315 10580 raft_consensus.cc:606] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Received request to transfer leadership to TS 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:20.812882 10580 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9042bec741aa476587972e688997da43"
dest_uuid: "33d77e3063684586a737bccf358786d8"
mode: GRACEFUL
new_leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:49158
I20260709 12:13:20.813834 10580 raft_consensus.cc:606] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Received request to transfer leadership to TS 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:20.847093 10506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
mode: GRACEFUL
new_leader_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:49114
I20260709 12:13:20.847991 10506 raft_consensus.cc:606] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Received request to transfer leadership to TS 33d77e3063684586a737bccf358786d8
I20260709 12:13:20.852552  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:20.855145  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:20.857074  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:20.857492 18510 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 13 to 15, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.868969 11218 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 13 to 15, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.876085 18565 ts_tablet_manager.cc:1929] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.13
I20260709 12:13:20.876731 18565 log.cc:1199] T 945dd7b821a5493f8a9d6ca251795da9 P 99d42e7288f145f1bd5af212fd062995: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/wals/945dd7b821a5493f8a9d6ca251795da9
I20260709 12:13:20.879514 10323 catalog_manager.cc:5028] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): tablet 945dd7b821a5493f8a9d6ca251795da9 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:13:20.907908 10582 raft_consensus.cc:2955] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 13 to 15, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.912147 10336 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 13 to 15, VOTER 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) evicted. New cstate: current_term: 22 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:20.925772 18571 tablet_replica.cc:333] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:20.927323 18571 raft_consensus.cc:2243] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:20.929816 10323 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c545de01b7fb4c89946871c1d522006f with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:20.930269 10494 tablet_service.cc:1558] Processing DeleteTablet for tablet c545de01b7fb4c89946871c1d522006f with delete_type TABLET_DATA_TOMBSTONED (TS 96a7b6a7b75246d381f5d4a71ea716d9 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:33776
I20260709 12:13:20.950883 18576 tablet_replica.cc:333] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:20.962052 18576 raft_consensus.cc:2243] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:20.953769 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49067.3us
I20260709 12:13:20.970750 18576 raft_consensus.cc:2272] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:20.975059 18576 ts_tablet_manager.cc:1916] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:20.995450 18571 raft_consensus.cc:2272] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:21.017196 18571 ts_tablet_manager.cc:1916] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:21.120718 18571 ts_tablet_manager.cc:1929] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 12:13:21.121392 18571 log.cc:1199] T 26d3d17d58be40c8974c348ca146a951 P 33d77e3063684586a737bccf358786d8: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/wals/26d3d17d58be40c8974c348ca146a951
I20260709 12:13:21.124337 18506 raft_consensus.cc:993] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: : Instructing follower 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:13:21.124459 18576 ts_tablet_manager.cc:1929] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.14
I20260709 12:13:21.125124 18506 raft_consensus.cc:1081] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Signalling peer 99d42e7288f145f1bd5af212fd062995 to start an election
I20260709 12:13:21.128017 10322 catalog_manager.cc:5028] TS 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): tablet 26d3d17d58be40c8974c348ca146a951 (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:13:21.131758 18577 raft_consensus.cc:993] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: : Instructing follower 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:13:21.153625 18577 raft_consensus.cc:1081] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Signalling peer 96a7b6a7b75246d381f5d4a71ea716d9 to start an election
I20260709 12:13:21.155483 10509 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9042bec741aa476587972e688997da43"
dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
 from {username='slave'} at 127.0.0.1:41084
I20260709 12:13:21.155644 18576 log.cc:1199] T c545de01b7fb4c89946871c1d522006f P 96a7b6a7b75246d381f5d4a71ea716d9: Deleting WAL directory at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/wals/c545de01b7fb4c89946871c1d522006f
I20260709 12:13:21.156121 10509 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.156423 10432 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7"
dest_uuid: "99d42e7288f145f1bd5af212fd062995"
 from {username='slave'} at 127.0.0.1:48986
I20260709 12:13:21.157043 10432 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.157423 10432 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:21.156618 10509 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:21.161770 10322 catalog_manager.cc:5028] TS 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): tablet c545de01b7fb4c89946871c1d522006f (table test-workload [id=044b427497c843d3931346454f4fb614]) successfully deleted
I20260709 12:13:21.190519 10509 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:21.200860 10582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:21.201737 10582 raft_consensus.cc:3055] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:13:21.202076 10582 raft_consensus.cc:740] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:21.203037 10582 consensus_queue.cc:260] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:21.204509 10582 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:21.197393 10432 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:21.211839 10432 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:21.214234 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:21.214953 10580 raft_consensus.cc:3055] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:21.215289 10580 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:21.216068 10580 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:21.217319 10580 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:21.212922 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:21.219769 10504 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:21.241935 10325 maintenance_manager.cc:419] P 92c087afe23c499b9e821ff6f85b7e2a: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033394
I20260709 12:13:21.261861 10509 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:21.265121 10504 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:13:21.274756 18479 raft_consensus.cc:993] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: : Instructing follower 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:21.279570 18479 raft_consensus.cc:1081] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Signalling peer 33d77e3063684586a737bccf358786d8 to start an election
I20260709 12:13:21.281939 10583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb"
dest_uuid: "33d77e3063684586a737bccf358786d8"
 from {username='slave'} at 127.0.0.1:55480
I20260709 12:13:21.282765 10583 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.283306 10583 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:21.289376 10403 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:21.290548 18515 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:13:21.291244 18515 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Becoming Leader. State: Replica: 99d42e7288f145f1bd5af212fd062995, State: Running, Role: LEADER
I20260709 12:13:21.292418 18515 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:21.303949 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:21.304862 11218 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:13:21.307652 10582 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:13:21.310068 10478 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:21.311213 18479 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:13:21.311861 18479 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:21.313048 18479 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:21.315846 10583 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:21.318219 10583 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:21.321194 10580 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99d42e7288f145f1bd5af212fd062995 in term 16.
I20260709 12:13:21.322319 10504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:21.324398 10432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "99d42e7288f145f1bd5af212fd062995"
I20260709 12:13:21.323123 10504 raft_consensus.cc:3055] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:21.325199 10504 raft_consensus.cc:740] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:21.325208 10432 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:21.343636 10504 consensus_queue.cc:260] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:21.355216 10504 raft_consensus.cc:3060] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:21.364354 10336 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 13 to 14, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 14 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.379876 10324 maintenance_manager.cc:643] P 92c087afe23c499b9e821ff6f85b7e2a: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.137s	user 0.049s	sys 0.004s Metrics: {"bytes_written":9631,"delete_count":0,"lbm_write_time_us":1212,"lbm_writes_lt_1ms":7,"reinsert_count":0,"update_count":353}
I20260709 12:13:21.383416 10432 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:21.384171 10325 maintenance_manager.cc:419] P 92c087afe23c499b9e821ff6f85b7e2a: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.422895
I20260709 12:13:21.385273 10553 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995; no voters: 
I20260709 12:13:21.386390 18577 raft_consensus.cc:2804] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:21.390372 18577 raft_consensus.cc:697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:21.391415 18577 consensus_queue.cc:237] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:21.392234 11218 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 14.
I20260709 12:13:21.429430 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: term changed from 15 to 16, leader changed from 33d77e3063684586a737bccf358786d8 (127.9.11.67) to 99d42e7288f145f1bd5af212fd062995 (127.9.11.65). New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.440557 10504 raft_consensus.cc:2468] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:21.466085 10328 catalog_manager.cc:5697] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.750348 10505 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:13:21.753706 18574 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:13:21.873309 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 48641.9us
I20260709 12:13:21.969820 10504 raft_consensus.cc:1275] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:21.987205 10583 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:13:21.988790 11218 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:13:22.025202 10580 raft_consensus.cc:1275] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:13:22.025593 18592 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:13:22.032091 18596 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:22.035164  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:22.090422 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 66062.8us
W20260709 12:13:22.091163 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 68098.2us
I20260709 12:13:22.099035 18580 consensus_queue.cc:1048] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:13:22.141149 10434 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
W20260709 12:13:22.148651 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 78363.6us
I20260709 12:13:22.107100 18479 consensus_queue.cc:1048] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 12:13:22.170043 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 92700.8us
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:461: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 2
I20260709 12:13:22.180001  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
W20260709 12:13:22.193447 10479 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
I20260709 12:13:22.263062 10504 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:13:22.290027 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 96217.5us
W20260709 12:13:22.291977 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 182205us
I20260709 12:13:22.282678 18510 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
W20260709 12:13:22.331223 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 45483.9us
W20260709 12:13:22.331214 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 71007.6us
W20260709 12:13:22.358932 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 90347.6us
W20260709 12:13:22.359457 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 85184.3us
I20260709 12:13:22.331635 10324 maintenance_manager.cc:643] P 92c087afe23c499b9e821ff6f85b7e2a: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.934s	user 0.130s	sys 0.042s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":29847,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"lbm_read_time_us":9243,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":19,"lbm_write_time_us":15844,"lbm_writes_1-10_ms":3,"lbm_writes_lt_1ms":9,"peak_mem_usage":589052,"reinsert_count":0,"thread_start_us":140513,"threads_started":2,"update_count":353}
W20260709 12:13:22.421686 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 51566.2us
W20260709 12:13:22.452695 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.468226 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:22.457427 10404 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4baf326f866a4a17b411837d7902a0c7. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.511433 18611 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:13:22.514443 10580 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Refusing update from remote peer 99d42e7288f145f1bd5af212fd062995: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:13:22.555846 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 179304us
W20260709 12:13:22.603147 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.627882 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.629396 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.630559 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.604411 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.631546 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.646715 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.664769 10553 consensus_peers.cc:597] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.668507 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 189074us
W20260709 12:13:22.667605 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.665620 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.672396 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.676159 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.672847 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.682021 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.686228 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 317107us
W20260709 12:13:22.684197 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.693012 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.706527 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.679539 18621 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:13:22.752224 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.756702 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 64698.3us
I20260709 12:13:22.768692 18600 raft_consensus.cc:2955] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:13:22.771771 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.772617 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.773283 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.773950 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.774505 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.776173 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.781037 10585 raft_consensus.cc:2955] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:13:22.783414 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.799734 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.800503 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.801084 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.802294 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.802930 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.842049 10509 raft_consensus.cc:2955] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
W20260709 12:13:22.884456 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:22.885088 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48785.4us
W20260709 12:13:22.894160 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 50383us
W20260709 12:13:22.899847 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 141839us
W20260709 12:13:22.914935 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 79760.6us
W20260709 12:13:23.000890 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49199.6us
W20260709 12:13:23.031625 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 94202.1us
I20260709 12:13:23.045887 18631 raft_consensus.cc:493] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.049754 18631 raft_consensus.cc:515] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.054592 18641 ts_tablet_manager.cc:933] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:13:23.056584 18596 raft_consensus.cc:493] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.060923 18596 raft_consensus.cc:515] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.057411 18621 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.064039 18621 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.066816 18635 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.067453 18635 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.070803 18621 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:23.086704 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 146573us
I20260709 12:13:23.092224 10584 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:23.097404 18596 leader_election.cc:290] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.107816 18615 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.109273 18635 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.110636 18631 leader_election.cc:290] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.079258 10335 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 reported cstate change: config changed from index -1 to 13, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 16 leader_uuid: "99d42e7288f145f1bd5af212fd062995" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:23.108558 18615 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.119370 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.121483 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "deab11eaf55b4f3ebe48926a1e2afa38" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.122037 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.125224 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.125738 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.138427 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.148165 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.154367 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "977ae790c1464057bdfdea6a1466fdc3" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.161373 18591 raft_consensus.cc:493] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.162200 18591 raft_consensus.cc:515] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.164830 18591 leader_election.cc:290] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.165441 18591 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.166383 18510 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.167107 18510 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.168983 18596 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.169476 18510 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.169629 18596 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.179085 18615 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.165978 18591 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.189014 18615 raft_consensus.cc:493] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.196398 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.196947 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.196885 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.201222 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.204561 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f94d710c36f04a9b8685258a7ff9d8a0" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.190701 18596 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.208570 18591 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.201768 18615 raft_consensus.cc:515] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.218629 18444 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.223737 18641 tablet_copy_client.cc:323] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.65:42153
I20260709 12:13:23.233623 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.239194 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90f71c8f44dc4ef79c01a6c9cd76cc61" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.241935 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.245663 10403 leader_election.cc:304] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.248191 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.238757 18591 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.248407 18621 raft_consensus.cc:2749] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:23.252331 18620 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:23.245429 18596 raft_consensus.cc:493] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.255839 10509 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:13:23.255983 18615 leader_election.cc:290] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.245258 18631 raft_consensus.cc:493] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.257836 10403 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.236625 18444 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.259202 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.267220 18444 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.269168 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.261102 18631 raft_consensus.cc:515] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.276819 18635 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:23.269012 18615 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.281364 18510 raft_consensus.cc:493] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.281600 18615 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.282055 18510 raft_consensus.cc:515] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.284464 18510 leader_election.cc:290] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.257283 18591 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.287142 18631 leader_election.cc:290] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.289332 18591 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.295238 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.297322 18506 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:13:23.300170 10403 leader_election.cc:304] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.301165 18591 raft_consensus.cc:2749] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:23.303623 18631 raft_consensus.cc:493] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.304440 18631 raft_consensus.cc:515] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.260684 18596 raft_consensus.cc:515] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.308188 18631 leader_election.cc:290] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.310334 10403 leader_election.cc:304] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.311408 18591 raft_consensus.cc:2749] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:23.313151 18615 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.314493 10403 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.316426 10403 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.317341 18591 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:23.317957 18615 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:23.318416 18596 leader_election.cc:290] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.320083 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.323551 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:23.324874 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90f71c8f44dc4ef79c01a6c9cd76cc61. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.326493 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.328842 18615 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:23.330518 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.332423 18615 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.340917 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.341962 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.342319 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63843cf9ccd4442cb52e3988597cabd8" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.342804 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.354813 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.356776 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.359125 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.360523 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.362545 18615 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.366894 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.373957 18631 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.373996 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bfb73cee8704df696d2eed4d27003bb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.380216 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a80f9fe1a0f840c4a6b9984e5fc64eed" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.380594 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.391601 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51452865e973445d9e13f232cfb5efb9" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.374856 18631 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.366255 18615 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.401739 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361eb2421d9548008ac1c2e32ba9ec14" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.396638 18604 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.406250 18444 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.425395 18631 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.427441 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.437574 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.439864 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81b70ca03fbe4622a03717efd079d598" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.440274 18444 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.448840 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.451122 10403 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.452044 18631 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:23.451208 18591 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:23.453625 10403 leader_election.cc:304] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.424263 18604 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.455184 10403 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.456388 18604 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.456319 18615 raft_consensus.cc:2749] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:23.456715 10403 leader_election.cc:304] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.456758 18510 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.459007 18635 raft_consensus.cc:2749] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:23.460620 10403 leader_election.cc:304] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.461447 18635 raft_consensus.cc:2749] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.464061 18635 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.464612 18635 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.468078 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.468456 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.476972 10403 leader_election.cc:304] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.484159 10505 raft_consensus.cc:2955] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:23.467553 18621 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.475689 18620 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.471325 18635 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.471844 18444 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.503496 18444 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.489610 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.481719 18588 raft_consensus.cc:2955] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:23.512468 18635 raft_consensus.cc:493] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.490271 18515 raft_consensus.cc:2749] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:23.496104 18621 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.526964 18621 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.521991 18620 raft_consensus.cc:493] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.528229 18620 raft_consensus.cc:515] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.530490 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.530820 18604 raft_consensus.cc:493] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:23.530540 18620 leader_election.cc:290] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.531507 18604 raft_consensus.cc:515] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.513144 18635 raft_consensus.cc:515] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.535652 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.538314 18510 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:13:23.538473 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 48957.3us
I20260709 12:13:23.539428 10403 leader_election.cc:304] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.540215 18631 raft_consensus.cc:2749] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:23.540808 18635 leader_election.cc:290] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.541553 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.544255 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.504238 18444 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:23.555183 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.554028 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73ade8620f2544acbb1e92193cf221b7" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.553890 18600 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:23.534264 18604 leader_election.cc:290] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.567921 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.570789 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.572827 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66985e55da4c4d3e88491e146eaaded2" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.582926 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f72c0bed7c45e484efbed50d2c364a" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.585626 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7919194a8a8542559be104753f043851" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.588029 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.583436 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "859a2c4bf0aa4585ad9a84c0c4fbdb2d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.584026 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3872158fef542f0b0f902e3a8756feb" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.586961 18604 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.589275 18600 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.593003 18600 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.595257 18600 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.596796 18510 raft_consensus.cc:493] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:23.597410 18510 raft_consensus.cc:515] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.592819 18604 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.602260 18604 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.605276 18444 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:23.606971 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:23.608754 18510 leader_election.cc:290] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.611884 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:23.618206 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.625373 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.629165 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.630599 10403 leader_election.cc:304] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.631904 18510 raft_consensus.cc:2749] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:23.632046 10403 leader_election.cc:304] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.634068 18604 raft_consensus.cc:2749] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:23.632443 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31117e61be004c52a161168a559023fd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.636384 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.637274 18444 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:23.637919 10403 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.638756 18444 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:23.639652 10403 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.640558 18510 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:23.641664 10403 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.642421 18510 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.642896 10403 leader_election.cc:304] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.643966 18510 raft_consensus.cc:2749] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:23.646750 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.651796 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.655925 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e3622c19ce24dc4a8573de044a7656b" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.661616 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.669929 18444 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:23.673228 10403 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.675076 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:23.678763 18604 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.694571 10403 leader_election.cc:304] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.686466 18510 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:23.698382 18604 raft_consensus.cc:2749] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:23.705848 10505 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:23.710644 18604 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.711427 18604 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.713840 18604 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.695549 18444 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:23.714740 18444 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:23.717515 18444 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.740454 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.743498 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5613e87b5a8c4948ae15672ea77f31cf" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.745508 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:23.747638 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e81a69eea26d461e9662b9b2aa4151cd" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:23.768350 10403 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.769369 18604 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:23.771021 10403 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:23.771904 18604 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:23.844929 10336 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 10, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:23.863823 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 448b86aa128243e48bed5f471ada874f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.879307 10580 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:13:23.880918 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:23.881549 18447 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:23.883034 18652 ts_tablet_manager.cc:933] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:23.935827  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:13:23.937754 10403 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 46913.5us
I20260709 12:13:23.951442  9261 tablet_replica.cc:333] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:23.954820  9261 raft_consensus.cc:2243] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:13:23.984786  9261 raft_consensus.cc:2272] T e8daad82d9f346a295fe1cf354383acc P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:23.997785  9261 tablet_replica.cc:333] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.001348  9261 raft_consensus.cc:2243] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.028324 18652 tablet_copy_client.cc:323] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:13:24.057751 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:13:24.058815 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 90f71c8f44dc4ef79c01a6c9cd76cc61 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-90f71c8f44dc4ef79c01a6c9cd76cc61
I20260709 12:13:24.083973  9261 raft_consensus.cc:2272] T 73ade8620f2544acbb1e92193cf221b7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.114252 10597 tablet_copy_source_session.cc:215] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:13:24.128125 18653 log.cc:927] Time spent T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:13:24.129628 18652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90f71c8f44dc4ef79c01a6c9cd76cc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:24.133821 18597 raft_consensus.cc:2955] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 LEADER]: Committing config change with OpId 25.16: config changed from index -1 to 16, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:24.144946 10584 raft_consensus.cc:2955] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index -1 to 16, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:24.163259  9261 tablet_replica.cc:333] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.164834  9261 raft_consensus.cc:2243] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.167470  9261 raft_consensus.cc:2272] T e81a69eea26d461e9662b9b2aa4151cd P 99d42e7288f145f1bd5af212fd062995 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.222047  9261 tablet_replica.cc:333] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.224433  9261 raft_consensus.cc:2243] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.232316  9261 raft_consensus.cc:2272] T 81b70ca03fbe4622a03717efd079d598 P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.232631 10584 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:24.236809 10328 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 16, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 25 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:24.248369  9261 tablet_replica.cc:333] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.250169  9261 raft_consensus.cc:2243] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.251895  9261 raft_consensus.cc:2272] T a082a11e47ba43b2acfaf230875b5e07 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.299669  9261 tablet_replica.cc:333] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.301601  9261 raft_consensus.cc:2243] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.303836 18652 tablet_copy_client.cc:806] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:24.304802 18652 tablet_copy_client.cc:670] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
W20260709 12:13:24.323218 10553 consensus_peers.cc:597] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 773a46c59e194295b8f5f7c5b6d58216. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:24.327648 10506 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:24.330152 18588 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:24.344980  9261 raft_consensus.cc:2272] T 0b474d9b241d4a698702287ee1bbe3b2 P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.354774 18652 tablet_copy_client.cc:538] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:24.361018 18656 ts_tablet_manager.cc:933] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:24.378355 10553 consensus_peers.cc:597] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:24.381089 18588 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Committing config change with OpId 22.17: config changed from index -1 to 17, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:24.424378 10509 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index -1 to 17, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } } }
I20260709 12:13:24.429426  9261 tablet_replica.cc:333] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.420845 18656 tablet_copy_client.cc:323] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.66:44681
I20260709 12:13:24.438625 10523 tablet_copy_service.cc:140] P 96a7b6a7b75246d381f5d4a71ea716d9: Received BeginTabletCopySession request for tablet 448b86aa128243e48bed5f471ada874f from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:38166)
I20260709 12:13:24.439692 10523 tablet_copy_service.cc:161] P 96a7b6a7b75246d381f5d4a71ea716d9: Beginning new tablet copy session on tablet 448b86aa128243e48bed5f471ada874f from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:38166: session id = 52bae2d2caf94d97b1049ecd4f82a2df-448b86aa128243e48bed5f471ada874f
I20260709 12:13:24.431671  9261 raft_consensus.cc:2243] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.454181 18652 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:13:24.474377  9261 raft_consensus.cc:2272] T 448b86aa128243e48bed5f471ada874f P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.487416 10328 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index -1 to 17, NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) added. New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:24.498135 10523 tablet_copy_source_session.cc:215] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:24.513232  9261 tablet_replica.cc:333] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.514922  9261 raft_consensus.cc:2243] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.539170  9261 raft_consensus.cc:2272] T b3872158fef542f0b0f902e3a8756feb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.592242 18656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 448b86aa128243e48bed5f471ada874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:24.609220  9261 tablet_replica.cc:333] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.612718  9261 raft_consensus.cc:2243] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.614212  9261 raft_consensus.cc:2272] T b4d2e9ee3d824817989d1fa5abcd9d67 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.629093  9261 tablet_replica.cc:333] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.630364  9261 raft_consensus.cc:2243] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.636399  9261 raft_consensus.cc:2272] T 7919194a8a8542559be104753f043851 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.645524  9261 tablet_replica.cc:333] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.658244 18656 tablet_copy_client.cc:806] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:24.659957  9261 raft_consensus.cc:2243] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.664871 18656 tablet_copy_client.cc:670] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
W20260709 12:13:24.667001 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.675149  9261 raft_consensus.cc:2272] T 58ffd928f77840c18d76b0b748c8ac26 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.679078 18656 tablet_copy_client.cc:538] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:13:24.679234 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.679266  9261 tablet_replica.cc:333] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.686159  9261 raft_consensus.cc:2243] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.688145  9261 raft_consensus.cc:2272] T 5613e87b5a8c4948ae15672ea77f31cf P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.696648  9261 tablet_replica.cc:333] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.703454  9261 raft_consensus.cc:2243] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.706375  9261 raft_consensus.cc:2272] T 51452865e973445d9e13f232cfb5efb9 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.710464  9261 tablet_replica.cc:333] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.716857  9261 raft_consensus.cc:2243] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.717849  9261 raft_consensus.cc:2272] T deab11eaf55b4f3ebe48926a1e2afa38 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:24.725622 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.735461  9261 tablet_replica.cc:333] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.737273  9261 raft_consensus.cc:2243] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.738153  9261 raft_consensus.cc:2272] T 361eb2421d9548008ac1c2e32ba9ec14 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.744315  9261 tablet_replica.cc:333] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
W20260709 12:13:24.752908 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.757228  9261 raft_consensus.cc:2243] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.758975  9261 raft_consensus.cc:2272] T cc34ac39d2184bffb94d60fe15003395 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.762939  9261 tablet_replica.cc:333] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.771941  9261 raft_consensus.cc:2243] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.773038  9261 raft_consensus.cc:2272] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.776911  9261 tablet_replica.cc:333] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.781327 18656 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
I20260709 12:13:24.787982  9261 raft_consensus.cc:2243] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.788928  9261 raft_consensus.cc:2272] T 66985e55da4c4d3e88491e146eaaded2 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:24.783425 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.783350 18652 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.792320 18652 tablet_bootstrap.cc:492] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:13:24.793390  9261 tablet_replica.cc:333] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.793394 18652 ts_tablet_manager.cc:1403] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.340s	user 0.046s	sys 0.005s
I20260709 12:13:24.800346 18652 raft_consensus.cc:359] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:24.801295 18652 raft_consensus.cc:740] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:24.802001 18652 consensus_queue.cc:260] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:24.803409  9261 raft_consensus.cc:2243] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.805294  9261 raft_consensus.cc:2272] T 9a78d11ad8304de6b3c842374e1a2e09 P 99d42e7288f145f1bd5af212fd062995 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:24.817523 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.821703 18652 ts_tablet_manager.cc:1434] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:13:24.825984  9261 tablet_replica.cc:333] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.827432  9261 raft_consensus.cc:2243] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:24.828998  9261 raft_consensus.cc:2272] T c545de01b7fb4c89946871c1d522006f P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.850282 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-90f71c8f44dc4ef79c01a6c9cd76cc61 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:13:24.851048 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-90f71c8f44dc4ef79c01a6c9cd76cc61 on tablet 90f71c8f44dc4ef79c01a6c9cd76cc61 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:24.851276  9261 tablet_replica.cc:333] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.858292 18663 ts_tablet_manager.cc:933] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:24.859261  9261 raft_consensus.cc:2243] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:24.860446 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:24.861295 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.866243  9261 raft_consensus.cc:2272] T f94d710c36f04a9b8685258a7ff9d8a0 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.870257  9261 tablet_replica.cc:333] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
W20260709 12:13:24.921615 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.931460 18663 tablet_copy_client.cc:323] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:13:24.933519 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 773a46c59e194295b8f5f7c5b6d58216 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:13:24.934281 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 773a46c59e194295b8f5f7c5b6d58216 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-773a46c59e194295b8f5f7c5b6d58216
I20260709 12:13:24.936595  9261 raft_consensus.cc:2243] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.937544  9261 raft_consensus.cc:2272] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.941118  9261 tablet_replica.cc:333] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.942030  9261 raft_consensus.cc:2243] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:24.955612 18604 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:24.956593 18604 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:24.960943 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:24.967795 18604 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:24.968055 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:24.969487 10597 tablet_copy_source_session.cc:215] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:24.971511 10404 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:24.969154 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:24.974750 18663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773a46c59e194295b8f5f7c5b6d58216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:24.975416 18604 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:24.978353 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:24.980542 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:24.988107 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:24.942787  9261 raft_consensus.cc:2272] T a80f9fe1a0f840c4a6b9984e5fc64eed P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:24.993749  9261 tablet_replica.cc:333] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:24.994724  9261 raft_consensus.cc:2243] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:25.021368 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.018754  9261 raft_consensus.cc:2272] T 2e3622c19ce24dc4a8573de044a7656b P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.032702 18604 raft_consensus.cc:493] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:25.033401 18604 raft_consensus.cc:515] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:25.036996 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.043726  9261 tablet_replica.cc:333] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.045895 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.046108 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c61f4bfa2db8497dbf7e0446e2ab707d" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.072937 10404 leader_election.cc:304] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:25.055125 18604 leader_election.cc:290] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.044855  9261 raft_consensus.cc:2243] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:25.068550 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.074113 18444 raft_consensus.cc:2749] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:25.074882  9261 raft_consensus.cc:2272] T 60f72c0bed7c45e484efbed50d2c364a P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:25.080948 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.082551 11212 raft_consensus.cc:1217] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
W20260709 12:13:25.087214 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:25.093273 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.095526  9261 tablet_replica.cc:333] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.096899 18452 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.097692 18452 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.100600 18452 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:25.105268 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.111687 18656 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:13:25.114367 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.115314 18656 tablet_bootstrap.cc:492] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:13:25.116514 18656 ts_tablet_manager.cc:1403] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.335s	user 0.051s	sys 0.019s
W20260709 12:13:25.116564 11178 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.117019  9261 raft_consensus.cc:2243] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.128458 18447 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.129050 18447 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.132818  9261 raft_consensus.cc:2272] T 977ae790c1464057bdfdea6a1466fdc3 P 99d42e7288f145f1bd5af212fd062995 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.140199 11177 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:25.147217 18452 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:25.151531 18447 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:25.153028 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:25.153767 10479 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.154341 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.155108 11212 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 23.
I20260709 12:13:25.159499 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:25.160481 18447 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:25.160916 18447 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.161398 18447 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:25.166221 18663 tablet_copy_client.cc:806] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
W20260709 12:13:25.169966 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.173564 18447 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.175801 18663 tablet_copy_client.cc:670] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:25.194398 18663 tablet_copy_client.cc:538] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:25.203187  9261 tablet_replica.cc:333] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.206404 18447 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.208292 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8daad82d9f346a295fe1cf354383acc" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:25.209430 11212 raft_consensus.cc:3060] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:25.228968 18604 raft_consensus.cc:493] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 52bae2d2caf94d97b1049ecd4f82a2df)
I20260709 12:13:25.229754 18604 raft_consensus.cc:515] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:25.219373 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:25.232261 10479 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.233937 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.233243 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8b322a475e40fcbca3445b6b0dcf26" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.232333 18604 leader_election.cc:290] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.221541  9261 raft_consensus.cc:2243] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:25.242018  9261 raft_consensus.cc:2272] T 6056d1f6a88d41d383e05a2c04508741 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.216625 18656 raft_consensus.cc:359] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:25.244442 18656 raft_consensus.cc:740] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:25.245368 18656 consensus_queue.cc:260] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:25.251132 10403 leader_election.cc:304] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:25.253371 18604 raft_consensus.cc:2749] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:25.255810 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.286129 18663 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:13:25.318164 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.323504 18656 ts_tablet_manager.cc:1434] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.206s	user 0.010s	sys 0.002s
I20260709 12:13:25.288115  9261 tablet_replica.cc:333] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.291765 11221 raft_consensus.cc:1217] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 LEARNER]: Deduplicated request from leader. Original: 25.15->[25.16-25.16]   Dedup: 25.16->[]
I20260709 12:13:25.326026 10523 tablet_copy_service.cc:342] P 96a7b6a7b75246d381f5d4a71ea716d9: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-448b86aa128243e48bed5f471ada874f received from {username='slave'} at 127.0.0.1:38166
W20260709 12:13:25.299589 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.326586 10523 tablet_copy_service.cc:434] P 96a7b6a7b75246d381f5d4a71ea716d9: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-448b86aa128243e48bed5f471ada874f on tablet 448b86aa128243e48bed5f471ada874f with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:25.334821  9261 raft_consensus.cc:2243] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:25.353243 18604 raft_consensus.cc:670] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:13:25.368475 11212 raft_consensus.cc:2468] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 24.
I20260709 12:13:25.370544 10479 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:25.380784 18447 raft_consensus.cc:2804] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:25.381541 18447 raft_consensus.cc:697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:25.393836  9261 raft_consensus.cc:2272] T 31117e61be004c52a161168a559023fd P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.382782 18667 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.397657 18447 consensus_queue.cc:237] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.389210 18444 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:25.435032 18444 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.449679 18444 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.422960 18667 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.450829 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.453231 18667 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:25.460115  9261 tablet_replica.cc:333] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.461570  9261 raft_consensus.cc:2243] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.463204  9261 raft_consensus.cc:2272] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:25.472750 11178 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.469808  9261 tablet_replica.cc:333] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.477623 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26d3d17d58be40c8974c348ca146a951" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.487422 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.489874 18669 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:25.490617 18669 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.492719 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:25.494318  9261 raft_consensus.cc:2243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.496171 10403 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:25.524829 10403 raft_consensus.cc:2686] T 26d3d17d58be40c8974c348ca146a951 P 99d42e7288f145f1bd5af212fd062995: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:13:25.535568 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:25.536659 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5d55f0ab8c4445abbbca950ea76dca1" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:25.547032 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 53574us
I20260709 12:13:25.561911 18660 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:25.539916 18669 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:25.565743 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 71562.7us
I20260709 12:13:25.568154  9261 raft_consensus.cc:2272] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.542852 18667 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.584033 18667 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.575636 10335 catalog_manager.cc:5697] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 23 to 24, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 24 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:25.586441 18667 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:25.589570 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.599853  9261 tablet_replica.cc:333] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.601022  9261 raft_consensus.cc:2243] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.601821  9261 raft_consensus.cc:2272] T 9bfb73cee8704df696d2eed4d27003bb P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.605523  9261 tablet_replica.cc:333] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.610044 10403 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:25.619045 18669 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:25.619809  9261 raft_consensus.cc:2243] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.627067  9261 raft_consensus.cc:2272] T f5d55f0ab8c4445abbbca950ea76dca1 P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.637365  9261 tablet_replica.cc:333] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
W20260709 12:13:25.656777 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.657574  9261 raft_consensus.cc:2243] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.658917  9261 raft_consensus.cc:2272] T 63843cf9ccd4442cb52e3988597cabd8 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.680518  9261 tablet_replica.cc:333] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.681704  9261 raft_consensus.cc:2243] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:13:25.685338 18674 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.686084 18674 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.690390 18678 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:25.690662 18680 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:25.691090 18678 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:25.693985 18672 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.694576 18672 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:25.696597 18604 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:25.696810 18672 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:25.697257 18604 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.698735 18669 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:25.699666 18604 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.691394 18680 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.703125 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.699452 18669 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:25.705150 18680 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.706465 18669 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:25.707212 18674 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.708426 11221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.709836 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:25.714502 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.711199 10584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7366ca68fce42f58f5e546ae5cad3ca" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:25.711434 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "147a37ae746d4efb84fdc2f6d4d5981e" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:25.712133 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62573c8e5a64ece95bc46d5fa18185c" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.715821  9261 raft_consensus.cc:2272] T 4baf326f866a4a17b411837d7902a0c7 P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.715286 11212 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:25.712683 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.713569 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05e004719a9442e2b9b045e4a6df5426" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:25.693601 18678 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:25.718842 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:25.719789 18674 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:25.720019  9261 tablet_replica.cc:333] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.720224 18674 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.720662 18674 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:25.721073  9261 raft_consensus.cc:2243] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.721845  9261 raft_consensus.cc:2272] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.722694 18659 raft_consensus.cc:1064] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
W20260709 12:13:25.725598 10479 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.729264 18659 consensus_queue.cc:237] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:25.735123 11178 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.735739 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:25.749905 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.752631 18678 raft_consensus.cc:493] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:25.753350 18678 raft_consensus.cc:515] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:25.751246 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.755764 18678 leader_election.cc:290] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:25.752758 18681 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:25.761096 10403 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:25.766716 10403 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:25.776400 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 195447us
I20260709 12:13:25.781081 18669 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:13:25.773804 18678 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:25.782519 10403 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:25.783461 18678 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:25.786892 10403 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:25.787704 10403 raft_consensus.cc:2686] T 05e004719a9442e2b9b045e4a6df5426 P 99d42e7288f145f1bd5af212fd062995: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:13:25.789793 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:25.790695 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:25.795769 10553 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.789826 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b117a8eaf2494d0ea2317e5895cac532" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.807247 18675 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.807977 18675 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:25.815881 10479 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.816864 18675 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.824106 18680 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:25.824848 18680 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:25.826737 10403 leader_election.cc:304] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:25.827082 18680 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:25.827733 18680 raft_consensus.cc:2749] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:25.832300 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:25.828465  9261 tablet_replica.cc:333] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.839634  9261 raft_consensus.cc:2243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.842221 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773a46c59e194295b8f5f7c5b6d58216" candidate_uuid: "99d42e7288f145f1bd5af212fd062995" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:25.843782 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:25.844764 18674 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.846976 10403 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99d42e7288f145f1bd5af212fd062995; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:25.850107 18674 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.850348  9261 raft_consensus.cc:2272] T 9f8b322a475e40fcbca3445b6b0dcf26 P 99d42e7288f145f1bd5af212fd062995 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:25.851989 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:25.851449 18680 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:25.869032 10479 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:25.872532 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:25.873378 11212 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 17.
I20260709 12:13:25.879895  9261 tablet_replica.cc:333] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.881019  9261 raft_consensus.cc:2243] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.881728  9261 raft_consensus.cc:2272] T c7366ca68fce42f58f5e546ae5cad3ca P 99d42e7288f145f1bd5af212fd062995 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.891630 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a082a11e47ba43b2acfaf230875b5e07" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:25.893393 11212 raft_consensus.cc:3060] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:25.903183 10506 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:13:25.917893 11221 raft_consensus.cc:1275] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:13:25.902709 11212 raft_consensus.cc:2468] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 20.
I20260709 12:13:25.923821 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:25.940969 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.946079 18671 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:25.946709 18671 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:25.947142 18671 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:25.948617  9261 tablet_replica.cc:333] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:25.949640  9261 raft_consensus.cc:2243] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.951176 18668 raft_consensus.cc:1064] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:13:25.951644  9261 raft_consensus.cc:2272] T b117a8eaf2494d0ea2317e5895cac532 P 99d42e7288f145f1bd5af212fd062995 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.955161 10479 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:25.987969 18670 raft_consensus.cc:2804] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:25.988834 18670 raft_consensus.cc:697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:25.989888 18670 consensus_queue.cc:237] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.954548 18668 consensus_queue.cc:237] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 14, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.959275 18671 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:25.994014 11212 raft_consensus.cc:1275] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:13:25.994517 18671 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:25.963466 10552 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:25.996682 18659 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:25.969237 18688 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:26.005684 18661 consensus_queue.cc:1048] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:26.006033 18659 consensus_queue.cc:1048] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:13:26.013734 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.005563 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b474d9b241d4a698702287ee1bbe3b2" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:26.014564 11212 raft_consensus.cc:3060] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:25.982829 18663 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:13:26.014602 10479 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.033861 11212 raft_consensus.cc:2468] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:26.035719 10479 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.044709 18688 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.045646 18688 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.048923 18688 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:26.054342 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:26.038280 18681 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.055147 18661 raft_consensus.cc:2804] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:26.056303 18681 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.064536 11218 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 15.
I20260709 12:13:26.065491 18661 raft_consensus.cc:697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:26.079442 18663 tablet_bootstrap.cc:492] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
I20260709 12:13:26.088555 18667 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.111039 18667 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.089962 10335 catalog_manager.cc:5697] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 19 to 20, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 20 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.091490 18681 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.092741 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:26.093698 18661 consensus_queue.cc:237] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.114511 10585 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:13:26.094414 18688 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.116452 18688 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.098244 18679 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.119371 18679 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.110811 18663 ts_tablet_manager.cc:1403] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.825s	user 0.060s	sys 0.010s
W20260709 12:13:26.105041 11178 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.130446 18679 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.151216 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.130512 18688 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:26.152232 18681 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:26.153614 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:26.153748 18681 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.154373 18681 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:13:26.154971 11178 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:13:26.155918 11178 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.156224 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:26.151928 18667 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.157748 11177 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.160389 18452 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:26.151034 10583 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:13:26.150667  9261 tablet_replica.cc:333] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.163844 18674 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.164065 11218 raft_consensus.cc:1275] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 LEARNER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:13:26.164451 18674 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.167052 18674 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.169742 18447 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.170338 18447 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:26.172433 18447 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
I20260709 12:13:26.172663 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:26.173305 10585 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:26.174091 11177 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.176528  9261 raft_consensus.cc:2243] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:26.178038  9261 raft_consensus.cc:2272] T c5a50eeaf1ac462883e185eddec889f6 P 99d42e7288f145f1bd5af212fd062995 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.180684 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:26.137956 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:13:26.181660 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 43863.2us
I20260709 12:13:26.172405 10509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:26.183657 10509 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 18.
I20260709 12:13:26.185382 10552 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:26.191211 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:26.187975 18659 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:26.200312 18659 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.200788 18659 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:26.215137 18675 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:26.215725 18675 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.216117 18675 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:26.217757 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.221195 18588 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:26.221757 18588 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.222230 18588 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:26.192090 10585 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
I20260709 12:13:26.228389 18671 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:13:26.188809 18667 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:26.243206  9261 tablet_replica.cc:333] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
W20260709 12:13:26.244095 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.247118 18663 raft_consensus.cc:359] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:26.248127 18663 raft_consensus.cc:740] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:26.248915 18663 consensus_queue.cc:260] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:26.244292  9261 raft_consensus.cc:2243] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:26.259958 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:26.260936 18663 ts_tablet_manager.cc:1434] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.140s	user 0.007s	sys 0.000s
I20260709 12:13:26.262406 18670 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:26.275171 18670 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.275704 18670 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:26.261510 18688 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.277455 18688 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.267221 18671 consensus_queue.cc:1048] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:26.268692 18682 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.279892 18682 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.282069 18682 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:26.222136 18659 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.282186 18688 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:26.289351 18659 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:26.287003 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.289389 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-773a46c59e194295b8f5f7c5b6d58216 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:13:26.291666 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-773a46c59e194295b8f5f7c5b6d58216 on tablet 773a46c59e194295b8f5f7c5b6d58216 with peer 52bae2d2caf94d97b1049ecd4f82a2df
I20260709 12:13:26.265853 18675 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.301435  9261 raft_consensus.cc:2272] T c882875c377649acb233035d47b8132e P 99d42e7288f145f1bd5af212fd062995 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.301591 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:26.302387 11218 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 4.
I20260709 12:13:26.306769 11217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:26.306952 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:26.307662 11218 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
W20260709 12:13:26.319368 10479 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.321396 18681 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.322875 11217 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 16: Already voted for candidate 52bae2d2caf94d97b1049ecd4f82a2df in this term.
I20260709 12:13:26.316983 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:26.325344 18676 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:26.325914 18676 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.326375 18676 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:26.328955 10335 catalog_manager.cc:5697] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 17 to 18, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 18 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.349879  9261 tablet_replica.cc:333] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
W20260709 12:13:26.359702 10479 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.362005  9261 raft_consensus.cc:2243] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:26.328845 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:26.364380 18659 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:26.364835 18659 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.365270 18659 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.370368 18688 raft_consensus.cc:2955] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:26.402237 11178 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.407191 18675 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:26.411597 10553 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:13:26.412709 10553 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.410440 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
W20260709 12:13:26.413635 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.414038 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:26.414355 10479 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.414911 18681 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.416061 18701 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:13:26.417443 11221 raft_consensus.cc:2955] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:26.421990 18670 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:26.430531 18670 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727), 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
W20260709 12:13:26.435407 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.436542  9261 raft_consensus.cc:2272] T c61f4bfa2db8497dbf7e0446e2ab707d P 99d42e7288f145f1bd5af212fd062995 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.438455 11212 raft_consensus.cc:1217] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEARNER]: Deduplicated request from leader. Original: 22.16->[22.17-22.17]   Dedup: 22.17->[]
I20260709 12:13:26.439452 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ffd928f77840c18d76b0b748c8ac26" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:26.440127 10580 raft_consensus.cc:3060] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:26.438565 10479 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.456964 18675 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.457593 18675 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.455490  9261 tablet_replica.cc:333] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.470007 10509 raft_consensus.cc:2955] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:26.474051 10479 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.472638 10335 catalog_manager.cc:5697] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 10 to 11, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:26.476861 18676 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.476795 18671 raft_consensus.cc:2955] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 LEADER]: Committing config change with OpId 25.17: config changed from index 16 to 17, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:26.478010 18588 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.479267 10585 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 96a7b6a7b75246d381f5d4a71ea716d9 in current term 19: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:26.482887 18588 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:26.487897 11218 raft_consensus.cc:2955] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 16 to 17, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:26.489980 11221 raft_consensus.cc:1275] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:26.493033 18696 consensus_queue.cc:1048] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:26.506073  9261 raft_consensus.cc:2243] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:26.506922  9261 raft_consensus.cc:2272] T 147a37ae746d4efb84fdc2f6d4d5981e P 99d42e7288f145f1bd5af212fd062995 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.508373 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:26.509407 18676 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:26.511121 10478 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.511610 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78d11ad8304de6b3c842374e1a2e09" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:26.512283 11212 raft_consensus.cc:3060] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:26.512342 18696 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:26.520251 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:26.521130 10506 raft_consensus.cc:2393] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 33d77e3063684586a737bccf358786d8 in current term 19: Already voted for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in this term.
I20260709 12:13:26.522794 18675 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.523850 10478 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.525619 18696 raft_consensus.cc:2749] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:26.466019 10580 raft_consensus.cc:2468] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:26.533660 10478 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.534493 18661 raft_consensus.cc:2804] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:26.535033 18661 raft_consensus.cc:697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:26.535943 18661 consensus_queue.cc:237] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:26.545092 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:26.546067 10583 raft_consensus.cc:2393] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 52bae2d2caf94d97b1049ecd4f82a2df in current term 16: Already voted for candidate 33d77e3063684586a737bccf358786d8 in this term.
I20260709 12:13:26.538790 11212 raft_consensus.cc:2468] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 5.
I20260709 12:13:26.551692 10583 raft_consensus.cc:2955] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Committing config change with OpId 25.17: config changed from index 16 to 17, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:26.555508 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.556059 10553 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.557641 10328 catalog_manager.cc:5697] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: config changed from index 16 to 17, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:26.560026  9261 tablet_replica.cc:333] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.561007  9261 raft_consensus.cc:2243] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 10 LEADER]: Raft consensus shutting down.
I20260709 12:13:26.561616 18676 raft_consensus.cc:2749] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:26.562330  9261 raft_consensus.cc:2272] T 4689962670ee421e8993be8a9ba7a657 P 99d42e7288f145f1bd5af212fd062995 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:26.564008 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.557279 18659 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.601154 18659 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:26.602138 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46674.2us
I20260709 12:13:26.608963 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:26.611002 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c545de01b7fb4c89946871c1d522006f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:26.611761 11218 raft_consensus.cc:3060] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.620078 11218 raft_consensus.cc:2468] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 23.
I20260709 12:13:26.623518 10553 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:26.639649 18676 raft_consensus.cc:2804] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:26.614341 18703 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:13:26.644197 18676 raft_consensus.cc:697] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:26.645349 18676 consensus_queue.cc:237] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.645244 10335 catalog_manager.cc:5697] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:26.661082 10553 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.719592  9261 tablet_replica.cc:333] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.720847  9261 raft_consensus.cc:2243] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:13:26.722589  9261 raft_consensus.cc:2272] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 99d42e7288f145f1bd5af212fd062995 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:26.728015 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 66985.2us
I20260709 12:13:26.728688 18672 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.730888 18672 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.742312 11218 raft_consensus.cc:1275] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:26.748399  9261 tablet_replica.cc:333] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.749480  9261 raft_consensus.cc:2243] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:26.750192  9261 raft_consensus.cc:2272] T 773a46c59e194295b8f5f7c5b6d58216 P 99d42e7288f145f1bd5af212fd062995 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.755540 10328 catalog_manager.cc:5697] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 4 to 5, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 5 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.769470 18672 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:26.770072 18675 consensus_queue.cc:1048] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:26.774098 10553 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:26.777360 18688 raft_consensus.cc:2804] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:26.778273 18688 raft_consensus.cc:697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 23 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:26.779124 18675 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.779747 18675 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:26.779388 18688 consensus_queue.cc:237] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.790338 10506 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:26.791150 10506 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 21.
W20260709 12:13:26.794003 10553 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.804658 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:26.782090 18675 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:26.806687 18672 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:26.807224 18672 raft_consensus.cc:493] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:26.807804 18672 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:26.824577 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
W20260709 12:13:26.826524 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:26.830693  9261 tablet_replica.cc:333] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.833946  9261 raft_consensus.cc:2243] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:26.834836  9261 raft_consensus.cc:2272] T d62573c8e5a64ece95bc46d5fa18185c P 99d42e7288f145f1bd5af212fd062995 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.875720 10328 catalog_manager.cc:5697] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 22 to 23, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 23 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.891230  9261 tablet_replica.cc:333] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995: stopping tablet replica
I20260709 12:13:26.892278  9261 raft_consensus.cc:2243] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 17 LEADER]: Raft consensus shutting down.
W20260709 12:13:26.917833 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.935310  9261 raft_consensus.cc:2272] T 4642a95b220c4ecea8115de1fa98954e P 99d42e7288f145f1bd5af212fd062995 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:26.918450 18672 raft_consensus.cc:515] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:26.955839 18672 leader_election.cc:290] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:26.963660 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.966900 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6056d1f6a88d41d383e05a2c04508741" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:26.967813 10505 raft_consensus.cc:3060] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:26.972162 18676 raft_consensus.cc:1064] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:13:26.975559 18676 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 15, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:26.991312 10505 raft_consensus.cc:2468] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 22.
I20260709 12:13:26.998023 10580 raft_consensus.cc:1275] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:26.999639 10552 leader_election.cc:304] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:27.000515 18672 raft_consensus.cc:2804] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:27.001884 18672 raft_consensus.cc:697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:27.002933 18672 consensus_queue.cc:237] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:27.010242 18675 consensus_queue.cc:1048] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:13:27.016563 10479 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.020049 10479 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.023340 10505 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:13:27.032503 18661 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:27.042050 11212 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:13:27.051565 18676 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:13:27.103446 10328 catalog_manager.cc:5697] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 21 to 22, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:27.108772 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 48136.2us
W20260709 12:13:27.117285 10553 leader_election.cc:336] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.121595 18676 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.124858 11221 raft_consensus.cc:1275] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:13:27.122346 18676 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:27.132279 18676 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:27.140061 18688 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:27.140635 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:27.141503 10510 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:27.142632 18688 consensus_queue.cc:1048] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:27.142882 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:27.143867 18676 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:27.144435 18676 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.144853 18676 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:27.157929 10553 consensus_peers.cc:597] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:27.193467 10553 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:13:27.195456 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:27.194625 10479 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 347 similar messages]
W20260709 12:13:27.196635 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.228981 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.243371 18588 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.244071 18588 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:27.246467 18588 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:27.249715 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:27.251281 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.254271 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.256064 18676 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:27.258625 18676 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:27.260129 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.262954 10553 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.263434 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.264106 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4baf326f866a4a17b411837d7902a0c7" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:27.264964 18659 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:27.266023 10510 raft_consensus.cc:3060] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:27.267266 10553 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:13:27.329456 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.330816 18588 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.331490 18588 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:27.335121 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:27.336220 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 50686.7us
W20260709 12:13:27.338075 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:27.346132 18588 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:27.361249 11212 raft_consensus.cc:1275] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:27.364285 18671 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.364962 18671 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:27.368520 18671 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:27.368566 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:27.369457 10583 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 18.
W20260709 12:13:27.370574 10479 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.371266 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.372252 18671 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:27.372722 18671 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.373518 18671 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:27.373828 10510 raft_consensus.cc:2468] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
I20260709 12:13:27.374516 18675 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.375015 18672 consensus_queue.cc:1048] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:27.375203 18675 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:27.377677 18675 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:27.379931 10552 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.381094 18682 raft_consensus.cc:2804] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:27.381589 18682 raft_consensus.cc:697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:27.382592 18682 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:27.384403 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:27.385133 10583 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 21.
I20260709 12:13:27.386617 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:27.387440 18675 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:27.387887 18675 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.388302 18675 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:27.387589 18659 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Committing config change with OpId 22.18: config changed from index 17 to 18, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:27.400255 10479 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.405943 18671 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:27.412549 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.416797 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5a50eeaf1ac462883e185eddec889f6" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:27.417527 10583 raft_consensus.cc:3060] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:27.423339 10479 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:13:27.425793 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 52445us
I20260709 12:13:27.426450 10510 raft_consensus.cc:1275] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:27.428150 18718 consensus_queue.cc:1048] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:27.436458 11212 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 17 to 18, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:27.425104 18675 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:27.426365 10583 raft_consensus.cc:2468] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 19.
I20260709 12:13:27.416955 18671 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:27.447841 18675 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:27.442699 10335 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 16 to 17, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:27.452075 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c882875c377649acb233035d47b8132e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:27.452733 10583 raft_consensus.cc:3060] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:27.451926 10509 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Committing config change with OpId 22.18: config changed from index 17 to 18, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:27.454604 10478 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.456240 18671 raft_consensus.cc:2804] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:27.456784 18671 raft_consensus.cc:697] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:27.457736 18671 consensus_queue.cc:237] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:27.486076 10479 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:13:27.496383 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.486075 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:27.507057 10583 raft_consensus.cc:2468] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a7b6a7b75246d381f5d4a71ea716d9 in term 22.
I20260709 12:13:27.508661 18672 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.509356 18672 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:27.511758 10478 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.513870 18672 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:27.519170 18671 raft_consensus.cc:2804] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:27.519992 18671 raft_consensus.cc:697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 LEADER]: Becoming Leader. State: Replica: 96a7b6a7b75246d381f5d4a71ea716d9, State: Running, Role: LEADER
I20260709 12:13:27.520920 18671 consensus_queue.cc:237] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:27.547765 10401 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.574643 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 50485.4us
I20260709 12:13:27.578920 10328 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 17 to 18, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:27.598090 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:27.619864 10583 consensus_queue.cc:237] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:27.622458 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.623190 10553 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.623601 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:27.628537 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.629405 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.629943 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:27.635042 18718 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:27.648742 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.664903 10510 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:13:27.677275 18676 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:13:27.630492 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.691311 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 60865.5us
W20260709 12:13:27.692397 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.702260 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.689374 18718 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Committing config change with OpId 22.19: config changed from index 18 to 19, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:27.703610 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.707186 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.711789 11218 raft_consensus.cc:1275] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:13:27.720007 10510 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Committing config change with OpId 22.19: config changed from index 18 to 19, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:27.726634 10335 catalog_manager.cc:5697] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 18 to 19, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 19 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:27.728569 10335 catalog_manager.cc:5697] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 reported cstate change: term changed from 21 to 22, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66). New cstate: current_term: 22 leader_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:27.743917 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.749348 10553 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:13:27.749871 10553 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:27.752555 18672 raft_consensus.cc:2749] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:13:27.758793 18588 consensus_queue.cc:1048] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
W20260709 12:13:27.849485 10553 connection.cc:769] client connection to 127.9.11.65:42153 send error: Network error: sendmsg error: Connection reset by peer (error 104)
W20260709 12:13:27.855671 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: sendmsg error: Connection reset by peer (error 104). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.854588 11178 connection.cc:573] client connection to 127.9.11.65:42153 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:13:27.887343 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.888015 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.888478 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.893041 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 124754us
I20260709 12:13:27.905766 10322 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 773a46c59e194295b8f5f7c5b6d58216 with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:27.922057 10336 catalog_manager.cc:5697] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 18 to 19, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New cstate: current_term: 22 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:27.928838 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:27.931455 10404 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 44506.1us
W20260709 12:13:27.943686 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.944864 11218 raft_consensus.cc:2955] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Committing config change with OpId 22.19: config changed from index 18 to 19, VOTER 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:27.962283 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:27.964864 18703 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:27.965559 18703 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:27.982613 18703 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:27.991858 11178 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:27.993978 10583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:27.994884 10583 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 10.
W20260709 12:13:27.995548 10479 consensus_peers.cc:597] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:27.998837 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:28.005272 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987): Couldn't send request to peer 52bae2d2caf94d97b1049ecd4f82a2df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4baf326f866a4a17b411837d7902a0c7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:28.005606 18667 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:13:28.006258 18667 raft_consensus.cc:493] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.006737 18667 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:13:28.017736 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:28.018922 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:28.020443 18723 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.021088 18723 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.023761 10505 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:13:28.025486 18672 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:13:28.027076 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:28.047699 18723 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:28.058857 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:28.068670 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:28.071162 10479 leader_election.cc:336] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.071686 10479 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:28.081598 18667 raft_consensus.cc:515] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.087157 18723 raft_consensus.cc:2749] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:13:28.105732 10583 raft_consensus.cc:1275] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:13:28.110950 18723 consensus_queue.cc:1048] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:28.130363 10583 raft_consensus.cc:1275] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Refusing update from remote peer 96a7b6a7b75246d381f5d4a71ea716d9: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:28.130286 10580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4689962670ee421e8993be8a9ba7a657" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:28.131130 10580 raft_consensus.cc:3060] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:13:28.132956 18667 leader_election.cc:290] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:28.139598 18727 consensus_queue.cc:1048] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:28.146420 10580 raft_consensus.cc:2468] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 11.
W20260709 12:13:28.147953 11178 leader_election.cc:336] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.150032 11177 leader_election.cc:304] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:28.152705 18667 raft_consensus.cc:2804] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:13:28.153231 18667 raft_consensus.cc:697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:28.154170 18667 consensus_queue.cc:237] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:28.199071 10323 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 89012.5us
W20260709 12:13:28.202890 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:28.197103 10479 consensus_peers.cc:597] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:28.260921 18737 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.261770 18737 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.277043 18737 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:28.243201 18742 log.cc:927] Time spent T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 12:13:28.300215 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61279.2us
W20260709 12:13:28.304481 11178 leader_election.cc:336] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.304626 10335 catalog_manager.cc:5697] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 10 to 11, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 11 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:28.329082 18727 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.329826 18727 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.332235 18727 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:28.345458 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:28.346248 10510 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 12.
I20260709 12:13:28.358944 11177 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:28.360770 18681 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:13:28.361225 18681 raft_consensus.cc:493] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.361603 18681 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:28.364449 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:28.375146 18667 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.375799 18667 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.379142 18667 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:28.415524 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
W20260709 12:13:28.416249 10479 leader_election.cc:336] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.416320 10510 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 17.
I20260709 12:13:28.418179 18681 raft_consensus.cc:515] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.433794 18681 leader_election.cc:290] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:28.434063 18729 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.436698 18729 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:28.439559 18729 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:28.443816 10479 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:28.448068 18723 raft_consensus.cc:2749] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:28.456018 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
I20260709 12:13:28.456964 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" is_pre_election: true
I20260709 12:13:28.457118 18681 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:28.457718 10510 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
I20260709 12:13:28.457813 18681 raft_consensus.cc:493] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting leader election (detected failure of leader 99d42e7288f145f1bd5af212fd062995)
I20260709 12:13:28.458317 18681 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:28.470866 18729 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.471509 18729 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.473433 10510 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fa6e3eaefd14aff89099fd64eeb1a1d" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:28.474109 10510 raft_consensus.cc:3060] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:13:28.474156 18729 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:28.515753 18681 raft_consensus.cc:515] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:28.528849 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:28.530720 11178 leader_election.cc:336] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.543921 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4642a95b220c4ecea8115de1fa98954e" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:28.544752 10505 raft_consensus.cc:3060] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:28.546195 18681 leader_election.cc:290] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:28.552683 10552 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:28.553799 18688 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:28.554338 18688 raft_consensus.cc:493] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.554838 18688 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:28.562809 10505 raft_consensus.cc:2468] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 18.
I20260709 12:13:28.571802 11218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:28.600488 11218 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 16.
I20260709 12:13:28.522897 10510 raft_consensus.cc:2468] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 13.
W20260709 12:13:28.599457 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 61772.5us
W20260709 12:13:28.566465 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:28.615927 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49499.4us
I20260709 12:13:28.635277 18688 raft_consensus.cc:515] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:28.580233 11178 leader_election.cc:336] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:28.639576 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:28.664420 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 84233.1us
I20260709 12:13:28.665361 11177 leader_election.cc:304] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:28.666654 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 63685.4us
W20260709 12:13:28.656196 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:28.673066 11177 leader_election.cc:304] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 
W20260709 12:13:28.699724 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 43599.7us
I20260709 12:13:28.668673 18667 raft_consensus.cc:2804] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:13:28.686172 10505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc34ac39d2184bffb94d60fe15003395" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "96a7b6a7b75246d381f5d4a71ea716d9"
I20260709 12:13:28.701431 18737 raft_consensus.cc:2804] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:28.701659 18667 raft_consensus.cc:697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:28.703126 18667 consensus_queue.cc:237] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.701984 10505 raft_consensus.cc:3060] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:28.690302 18688 leader_election.cc:290] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:28.713585 11178 leader_election.cc:336] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.717625 18641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57890 (request call id 2010) took 5809 ms. Trace:
I20260709 12:13:28.721683 18641 rpcz_store.cc:276] 0709 12:13:22.908217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.908360 (+   143us) service_pool.cc:224] Handling call
0709 12:13:23.059025 (+150665us) ts_tablet_manager.cc:934] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153)
0709 12:13:28.717229 (+5658204us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":21421,"tablet-copy.queue_time_us":124811,"thread_start_us":194079,"threads_started":2}
W20260709 12:13:28.723691 11176 connection.cc:466] server connection from 127.0.0.1:57890 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:57890 (request call id 2010) could send its response
I20260709 12:13:28.724198 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:28.734539 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:28.738962 18729 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:28.742390 18729 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.742844 18729 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:28.754477 10553 leader_election.cc:336] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:28.760699 10505 raft_consensus.cc:2468] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:28.763274 10552 leader_election.cc:304] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9; no voters: 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:28.782382 10582 raft_consensus.cc:1275] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:13:28.779271 18737 raft_consensus.cc:697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:28.790354 18667 consensus_queue.cc:1048] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:28.779078 18672 raft_consensus.cc:2804] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:13:28.814998 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:28.831738 18729 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:28.881861 18672 raft_consensus.cc:697] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:28.855160 18729 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Requested vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:28.922323 18672 consensus_queue.cc:237] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:28.941639 10478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 57320us
I20260709 12:13:28.968256 11212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4d2e9ee3d824817989d1fa5abcd9d67" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:28.969214 11212 raft_consensus.cc:3060] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:28.932472 18737 consensus_queue.cc:237] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:29.038755 11212 raft_consensus.cc:2468] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 17.
W20260709 12:13:29.041744 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 107717us
W20260709 12:13:29.053229  9261 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:29.097108 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:29.119127 10553 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:29.136972 10336 catalog_manager.cc:5697] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 12 to 13, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 13 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.139207 10336 catalog_manager.cc:5697] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 17 to 18, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 18 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.157608 18766 raft_consensus.cc:2804] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:29.159864 18766 raft_consensus.cc:697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:29.161116 18766 consensus_queue.cc:237] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:29.166320 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:29.176442 10553 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:29.177397 10335 catalog_manager.cc:5697] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 18 to 20, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.195880 10510 raft_consensus.cc:1275] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:13:29.225677 18681 consensus_queue.cc:1048] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:13:29.335189 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:29.337910 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:29.502348 10509 raft_consensus.cc:1275] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:29.506083 18751 consensus_queue.cc:1048] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:29.595341 10509 raft_consensus.cc:1275] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Refusing update from remote peer 52bae2d2caf94d97b1049ecd4f82a2df: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:13:29.619843 10553 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:29.622292 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:29.638526 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:29.640167 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:13:29.652325 18681 consensus_queue.cc:1048] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:29.711409 18783 ts_tablet_manager.cc:933] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Initiating tablet copy from peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:29.713433 10553 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:29.743319 11212 raft_consensus.cc:1275] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:13:29.747689 10336 catalog_manager.cc:5697] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 15 to 17, leader changed from 99d42e7288f145f1bd5af212fd062995 (127.9.11.65) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:29.751683 10323 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 113634us
W20260709 12:13:29.763190 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:13:29.767481 18754 consensus_queue.cc:1048] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:13:29.793691 18783 tablet_copy_client.cc:323] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Beginning tablet copy session from remote peer at address 127.9.11.67:36727
I20260709 12:13:29.795791 10597 tablet_copy_service.cc:140] P 33d77e3063684586a737bccf358786d8: Received BeginTabletCopySession request for tablet 4baf326f866a4a17b411837d7902a0c7 from peer 52bae2d2caf94d97b1049ecd4f82a2df ({username='slave'} at 127.0.0.1:36746)
I20260709 12:13:29.796391 10597 tablet_copy_service.cc:161] P 33d77e3063684586a737bccf358786d8: Beginning new tablet copy session on tablet 4baf326f866a4a17b411837d7902a0c7 from peer 52bae2d2caf94d97b1049ecd4f82a2df at {username='slave'} at 127.0.0.1:36746: session id = 52bae2d2caf94d97b1049ecd4f82a2df-4baf326f866a4a17b411837d7902a0c7
W20260709 12:13:29.811810 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:13:29.857378 10597 tablet_copy_source_session.cc:215] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:29.874130 18783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4baf326f866a4a17b411837d7902a0c7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:13:29.885237 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:29.886126 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:29.918322 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:29.921087 18783 tablet_copy_client.cc:806] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:29.921957 18783 tablet_copy_client.cc:670] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:29.931422 18783 tablet_copy_client.cc:538] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:30.005008 18783 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap starting.
W20260709 12:13:30.024803 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.049374 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.053820  9261 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:30.085373 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.092100 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:30.086300 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.116623 18783 log.cc:927] Time spent T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:13:30.140023 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:13:30.150710 18783 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:30.152215 18783 tablet_bootstrap.cc:492] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Bootstrap complete.
W20260709 12:13:30.152328 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:30.153261 18783 ts_tablet_manager.cc:1403] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent bootstrapping tablet: real 0.149s	user 0.050s	sys 0.008s
I20260709 12:13:30.158650 18783 raft_consensus.cc:359] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:30.159843 18783 raft_consensus.cc:740] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Initialized, Role: LEARNER
I20260709 12:13:30.162055 18783 consensus_queue.cc:260] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: NON_VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: true } }
I20260709 12:13:30.165249 18783 ts_tablet_manager.cc:1434] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:13:30.175834 10597 tablet_copy_service.cc:342] P 33d77e3063684586a737bccf358786d8: Request end of tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4baf326f866a4a17b411837d7902a0c7 received from {username='slave'} at 127.0.0.1:36746
I20260709 12:13:30.176306 10597 tablet_copy_service.cc:434] P 33d77e3063684586a737bccf358786d8: ending tablet copy session 52bae2d2caf94d97b1049ecd4f82a2df-4baf326f866a4a17b411837d7902a0c7 on tablet 4baf326f866a4a17b411837d7902a0c7 with peer 52bae2d2caf94d97b1049ecd4f82a2df
W20260709 12:13:30.182765 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.238569 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:30.251036 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:30.255978 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.274544 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.281384 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.285300 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.327875 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.348502 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.355196 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.361815 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.371523 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.376477 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.388598 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.391305 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.410367 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.411537 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.515508 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:13:30.570129 11218 raft_consensus.cc:1217] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Deduplicated request from leader. Original: 16.13->[17.14-17.14]   Dedup: 17.14->[]
W20260709 12:13:30.667845 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:30.706599 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:30.832561 10479 consensus_peers.cc:597] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:30.861613 10479 consensus_peers.cc:597] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:31.054329  9261 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:31.053903 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:31.077450 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:31.258826 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 46674.6us
I20260709 12:13:31.289995 18672 raft_consensus.cc:1064] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: attempting to promote NON_VOTER 52bae2d2caf94d97b1049ecd4f82a2df to VOTER
I20260709 12:13:31.293521 18672 consensus_queue.cc:237] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 13, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:31.355659 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 51510.2us
W20260709 12:13:31.379562 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:31.407970 11218 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEARNER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:13:31.415901 18748 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:31.416848 10510 raft_consensus.cc:1275] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Refusing update from remote peer 33d77e3063684586a737bccf358786d8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:13:31.418977 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 54024.2us
I20260709 12:13:31.447304 18753 consensus_queue.cc:1048] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:13:31.459764 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:31.509510 18792 raft_consensus.cc:2955] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Committing config change with OpId 17.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:31.513116 10510 raft_consensus.cc:2955] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
W20260709 12:13:31.522181 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:31.536033 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:13:31.539069 11218 raft_consensus.cc:2955] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } } }
I20260709 12:13:31.539245 10336 catalog_manager.cc:5697] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 reported cstate change: config changed from index 13 to 15, 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:31.601540 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:31.657306 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:31.949730 10553 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:32.054840  9261 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:32.173236 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.183151 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.230648 11178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111) [suppressed 438 similar messages]
W20260709 12:13:32.265625 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.347745 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:32.354781 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.404557 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:32.450692 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.455839 10553 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:32.468132 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.468827 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.510085 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.526924 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.597252 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.693924 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:32.728880 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:32.755352 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:32.758288 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:33.055550  9261 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:33.191098 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:33.191864 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.192466 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.193039 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.217587 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.218261 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.218897 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.229591 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.252693 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.253002 10323 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 45771.5us
W20260709 12:13:33.274597 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.275463 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:33.283420 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 43943.1us
W20260709 12:13:33.312925 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:33.335819 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 118748us
W20260709 12:13:33.337522 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47724.3us
W20260709 12:13:33.376891 10552 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 77519.6us
W20260709 12:13:33.406591 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.415482 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.428125 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 130189us
W20260709 12:13:33.510144 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.527671 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.545383 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.565047 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.599516 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:33.741329 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:33.752521 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:34.025389 10479 consensus_peers.cc:597] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:34.026665 10479 consensus_peers.cc:597] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:34.077406 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08003072c0 after lost signal to thread 9264
W20260709 12:13:34.080670 10608 debug-util.cc:398] Leaking SignalData structure 0x7b0800276180 after lost signal to thread 10370
W20260709 12:13:34.082526 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08003496a0 after lost signal to thread 10537
W20260709 12:13:34.084823 10608 debug-util.cc:398] Leaking SignalData structure 0x7b0800153420 after lost signal to thread 10611
W20260709 12:13:34.086488 10608 debug-util.cc:398] Leaking SignalData structure 0x7b08003a5620 after lost signal to thread 11248
W20260709 12:13:34.160742 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 75698.7us
W20260709 12:13:34.206205  9261 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:34.227180 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 96005.7us
W20260709 12:13:34.416105 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:34.416813 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 104282us
W20260709 12:13:34.420665 10479 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 49648.7us
W20260709 12:13:34.494563 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:34.524715 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:34.558470 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:34.583925 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:34.627843 10553 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:34.650763 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:34.837771 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:34.856786 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:34.884197 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:34.889968 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9a80 after lost signal to thread 9264
W20260709 12:13:34.892066 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9a60 after lost signal to thread 10370
W20260709 12:13:34.894366 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08002356a0 after lost signal to thread 10537
W20260709 12:13:34.897449 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf420 after lost signal to thread 10611
W20260709 12:13:34.902215 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8fa0 after lost signal to thread 11248
W20260709 12:13:34.902854 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001f66c0 after lost signal to thread 18793
W20260709 12:13:34.907598 10534 debug-util.cc:398] Leaking SignalData structure 0x7b08001fc7a0 after lost signal to thread 18812
W20260709 12:13:34.935482 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.015036 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.078974 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.126063 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:35.138370 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.166044 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.207479  9261 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:35.210351 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:35.228675 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.254364 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.300544 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.335434 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.352116 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:35.410970 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:35.460183 10553 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:35.478314 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.557047 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.558693 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.642932 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.654193 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.696216 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.737830 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:35.769734 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.788460 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.794682 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.858144 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:35.946883 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.987417 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:35.995934 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:36.101580 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:36.171393 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:36.183714 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:36.204052 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:36.208992  9261 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:36.314495 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:36.345613 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:36.519227 10479 consensus_peers.cc:597] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:36.589895 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:13:36.718027 10479 consensus_peers.cc:597] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:36.751642 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:36.935712 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:36.985242 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:37.112043 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:37.210228  9261 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:37.213694 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.218623 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:37.233219 10479 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111) [suppressed 443 similar messages]
W20260709 12:13:37.355587 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.365437 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.399995 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:37.410607 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.421290 10553 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:37.491801 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:37.515529 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:37.530756 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.560034 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:37.620256 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:37.633991 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.637600 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.661245 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.870056 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 61184.4us
W20260709 12:13:37.982069 10553 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:37.990737 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:37.994386 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.030798 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:38.049909 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.033329 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.070148 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.125468 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.210893  9261 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:38.211915 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.234643 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:38.263176 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.273947 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.279668 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.345167 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:38.379288 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.410586 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.423154 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:38.508332 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:38.531973 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.539965 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.561553 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.723332 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.731357 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.737669 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:38.974617 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:39.211530  9261 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:39.215196 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:39.278661 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:13:39.313259 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:36727, user_credentials={real_user=slave}} blocked reactor thread for 106550us
W20260709 12:13:39.439635 10479 consensus_peers.cc:597] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:39.486943 10479 consensus_peers.cc:597] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:39.491712 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:39.528254 10553 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:39.609277 10479 consensus_peers.cc:597] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:39.650092 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:39.828310 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:39.835827 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:39.874882 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:39.876421 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:39.897066 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.016546 10479 consensus_peers.cc:597] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:40.020043 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.030946 10479 consensus_peers.cc:597] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:40.061269 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.175874 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.188786 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.212121  9261 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 10459)
W20260709 12:13:40.223515 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:40.344873 10479 consensus_peers.cc:597] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:40.394723 10479 consensus_peers.cc:597] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.436748 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.437461 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.497498 10553 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:13:40.505668 10479 consensus_peers.cc:597] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.555670 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:40.556404 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.634464 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.643818 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.746608 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.757375 10479 consensus_peers.cc:597] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.843161 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:40.854884 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.862574 10479 consensus_peers.cc:597] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.918952 10479 consensus_peers.cc:597] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:40.924988 10479 consensus_peers.cc:597] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:40.980405 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:13:41.005259  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
W20260709 12:13:41.065169 10479 consensus_peers.cc:597] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:41.103132 10479 consensus_peers.cc:597] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:41.121726 10479 consensus_peers.cc:597] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:41.139669 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:13:41.343065  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
W20260709 12:13:41.357882 10479 consensus_peers.cc:597] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:41.373968 10553 consensus_peers.cc:597] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:41.380326 11177 consensus_peers.cc:597] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.388211 10552 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.404333 10479 consensus_peers.cc:597] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:13:41.418900 10552 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.433228 10552 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.447083 10552 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.449419 10552 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.451310 10552 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.455783 10552 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.468057 11177 consensus_peers.cc:597] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.474364  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:13:41.478076  9261 tablet_replica.cc:333] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.481105  9261 raft_consensus.cc:2243] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.482719  9261 raft_consensus.cc:2272] T 73ade8620f2544acbb1e92193cf221b7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.486409  9261 tablet_replica.cc:333] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.487471  9261 raft_consensus.cc:2243] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.491401  9261 raft_consensus.cc:2272] T e8daad82d9f346a295fe1cf354383acc P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.497181  9261 tablet_replica.cc:333] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.498234  9261 raft_consensus.cc:2243] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.499658  9261 raft_consensus.cc:2272] T 81b70ca03fbe4622a03717efd079d598 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.502923  9261 tablet_replica.cc:333] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.503840  9261 raft_consensus.cc:2243] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.505183  9261 raft_consensus.cc:2272] T e81a69eea26d461e9662b9b2aa4151cd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.508102  9261 tablet_replica.cc:333] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.515128  9261 raft_consensus.cc:2243] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.516463  9261 raft_consensus.cc:2272] T 945dd7b821a5493f8a9d6ca251795da9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.517370 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:13:41.519418  9261 tablet_replica.cc:333] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.520468  9261 raft_consensus.cc:2243] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.521377  9261 raft_consensus.cc:2272] T 38ead18e1a814c8b958f80396faa3170 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.524361  9261 tablet_replica.cc:333] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.525322  9261 raft_consensus.cc:2243] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Raft consensus shutting down.
W20260709 12:13:41.525725 10479 consensus_peers.cc:597] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:13:41.526722  9261 raft_consensus.cc:2272] T a082a11e47ba43b2acfaf230875b5e07 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.529965  9261 tablet_replica.cc:333] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.530864  9261 raft_consensus.cc:2243] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.534801  9261 raft_consensus.cc:2272] T 0b474d9b241d4a698702287ee1bbe3b2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.537801  9261 tablet_replica.cc:333] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.544710  9261 raft_consensus.cc:2243] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.554289  9261 raft_consensus.cc:2272] T 448b86aa128243e48bed5f471ada874f P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.557592  9261 tablet_replica.cc:333] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.561510  9261 raft_consensus.cc:2243] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.564531  9261 raft_consensus.cc:2272] T b3872158fef542f0b0f902e3a8756feb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.568114 11177 consensus_peers.cc:597] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.568962  9261 tablet_replica.cc:333] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.570008  9261 raft_consensus.cc:2243] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.571444  9261 raft_consensus.cc:2272] T 58ffd928f77840c18d76b0b748c8ac26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.574724  9261 tablet_replica.cc:333] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.576828  9261 raft_consensus.cc:2243] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.577524  9261 raft_consensus.cc:2272] T 1ecefe266eea48c4af129b425f391fe9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.580807  9261 tablet_replica.cc:333] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.581784  9261 raft_consensus.cc:2243] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.582497  9261 raft_consensus.cc:2272] T f94d710c36f04a9b8685258a7ff9d8a0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.585726  9261 tablet_replica.cc:333] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.587167  9261 raft_consensus.cc:2243] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.588197  9261 raft_consensus.cc:2272] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.601421 10552 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.591516  9261 tablet_replica.cc:333] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.604000  9261 raft_consensus.cc:2243] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.604761  9261 raft_consensus.cc:2272] T a80f9fe1a0f840c4a6b9984e5fc64eed P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.607884  9261 tablet_replica.cc:333] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.608785  9261 raft_consensus.cc:2243] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.613927  9261 raft_consensus.cc:2272] T 2e3622c19ce24dc4a8573de044a7656b P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.617080  9261 tablet_replica.cc:333] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.618093  9261 raft_consensus.cc:2243] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.619410  9261 raft_consensus.cc:2272] T 5613e87b5a8c4948ae15672ea77f31cf P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.614432 10552 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.622536  9261 tablet_replica.cc:333] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
W20260709 12:13:41.627648 11177 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.631848 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:41.632719 11177 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.633427 11177 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.634230 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:41.631932  9261 raft_consensus.cc:2243] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:41.649093 11177 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.649889  9261 raft_consensus.cc:2272] T 51452865e973445d9e13f232cfb5efb9 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.629094 10552 consensus_peers.cc:597] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.653211  9261 tablet_replica.cc:333] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.654158  9261 raft_consensus.cc:2243] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.654855  9261 raft_consensus.cc:2272] T 361eb2421d9548008ac1c2e32ba9ec14 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.658975  9261 tablet_replica.cc:333] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.659946  9261 raft_consensus.cc:2243] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.661290  9261 raft_consensus.cc:2272] T cfb1df1b6ba3414899a137b410511b26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.664381 11177 consensus_peers.cc:597] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.666479 10552 consensus_peers.cc:597] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.668679  9261 tablet_replica.cc:333] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.669842  9261 raft_consensus.cc:2243] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.671304  9261 raft_consensus.cc:2272] T e4cd5d330f694dacb199bd61266c8f5a P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.674433  9261 tablet_replica.cc:333] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
W20260709 12:13:41.673393 10552 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.676016 11177 consensus_peers.cc:597] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.677461 10552 consensus_peers.cc:597] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.675450  9261 raft_consensus.cc:2243] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.679307  9261 raft_consensus.cc:2272] T bea092b9128d46b6a42f4fe9aa23c739 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.682518  9261 tablet_replica.cc:333] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.683383  9261 raft_consensus.cc:2243] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.684015  9261 raft_consensus.cc:2272] T a0bb77704fc9403481da02083315dbb1 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.687160  9261 tablet_replica.cc:333] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.687969  9261 raft_consensus.cc:2243] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.689350  9261 raft_consensus.cc:2272] T 66985e55da4c4d3e88491e146eaaded2 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.692335  9261 tablet_replica.cc:333] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.693115  9261 raft_consensus.cc:2243] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.693928  9261 raft_consensus.cc:2272] T cc34ac39d2184bffb94d60fe15003395 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.696856  9261 tablet_replica.cc:333] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.697690  9261 raft_consensus.cc:2243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.700966  9261 raft_consensus.cc:2272] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.704945  9261 tablet_replica.cc:333] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.705862  9261 raft_consensus.cc:2243] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.706683  9261 raft_consensus.cc:2272] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.709731  9261 tablet_replica.cc:333] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.710511  9261 raft_consensus.cc:2243] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.711339  9261 raft_consensus.cc:2272] T 977ae790c1464057bdfdea6a1466fdc3 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.714545  9261 tablet_replica.cc:333] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.715377  9261 raft_consensus.cc:2243] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.716027  9261 raft_consensus.cc:2272] T c61f4bfa2db8497dbf7e0446e2ab707d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.716403 11177 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.719480  9261 tablet_replica.cc:333] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.720407  9261 raft_consensus.cc:2243] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.721318  9261 raft_consensus.cc:2272] T 6056d1f6a88d41d383e05a2c04508741 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.724823  9261 tablet_replica.cc:333] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.725663  9261 raft_consensus.cc:2243] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.726964  9261 raft_consensus.cc:2272] T 31117e61be004c52a161168a559023fd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.729802  9261 tablet_replica.cc:333] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.730594  9261 raft_consensus.cc:2243] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.731886  9261 raft_consensus.cc:2272] T 26d3d17d58be40c8974c348ca146a951 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.734936  9261 tablet_replica.cc:333] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.735723  9261 raft_consensus.cc:2243] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.736334  9261 raft_consensus.cc:2272] T dc98cd5edc9240fba53978ed59191cbd P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.739100  9261 tablet_replica.cc:333] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.739853  9261 raft_consensus.cc:2243] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.740459  9261 raft_consensus.cc:2272] T 06d2724d97aa43298ebaf9e746580fd0 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.744227  9261 tablet_replica.cc:333] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.745031  9261 raft_consensus.cc:2243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.746640  9261 raft_consensus.cc:2272] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.749500  9261 tablet_replica.cc:333] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.750285  9261 raft_consensus.cc:2243] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.750922  9261 raft_consensus.cc:2272] T 9bfb73cee8704df696d2eed4d27003bb P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.754894  9261 tablet_replica.cc:333] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.755648  9261 raft_consensus.cc:2243] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.756927  9261 raft_consensus.cc:2272] T 05e004719a9442e2b9b045e4a6df5426 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.767509 10479 consensus_peers.cc:597] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:13:41.769387  9261 tablet_replica.cc:333] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.770287  9261 raft_consensus.cc:2243] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.771440  9261 raft_consensus.cc:2272] T 4baf326f866a4a17b411837d7902a0c7 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.774521  9261 tablet_replica.cc:333] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.775403  9261 raft_consensus.cc:2243] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.776695  9261 raft_consensus.cc:2272] T c882875c377649acb233035d47b8132e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.780107  9261 tablet_replica.cc:333] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.780905  9261 raft_consensus.cc:2243] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.782182  9261 raft_consensus.cc:2272] T 9042bec741aa476587972e688997da43 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.785079  9261 tablet_replica.cc:333] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.785838  9261 raft_consensus.cc:2243] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.786543  9261 raft_consensus.cc:2272] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 96a7b6a7b75246d381f5d4a71ea716d9 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.789079 10552 consensus_peers.cc:597] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:41.790304  9261 tablet_replica.cc:333] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.791206  9261 raft_consensus.cc:2243] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.791824  9261 raft_consensus.cc:2272] T b117a8eaf2494d0ea2317e5895cac532 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.796193  9261 tablet_replica.cc:333] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.796954  9261 raft_consensus.cc:2243] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.798372  9261 raft_consensus.cc:2272] T c5a50eeaf1ac462883e185eddec889f6 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.801409  9261 tablet_replica.cc:333] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.802202  9261 raft_consensus.cc:2243] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:41.804718  9261 raft_consensus.cc:2272] T d62573c8e5a64ece95bc46d5fa18185c P 96a7b6a7b75246d381f5d4a71ea716d9 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.813354  9261 tablet_replica.cc:333] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.814242  9261 raft_consensus.cc:2243] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:41.816816 10553 consensus_peers.cc:597] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:13:41.817911  9261 raft_consensus.cc:2272] T 4642a95b220c4ecea8115de1fa98954e P 96a7b6a7b75246d381f5d4a71ea716d9 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.821538  9261 tablet_replica.cc:333] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.822678  9261 raft_consensus.cc:2243] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.824077  9261 raft_consensus.cc:2272] T 773a46c59e194295b8f5f7c5b6d58216 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.827101  9261 tablet_replica.cc:333] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.828011  9261 raft_consensus.cc:2243] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.828617  9261 raft_consensus.cc:2272] T 80de1e7bce9c4d34b0583315cafbc371 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:41.831940  9261 tablet_replica.cc:333] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9: stopping tablet replica
I20260709 12:13:41.832602  9261 raft_consensus.cc:2243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:41.833422  9261 raft_consensus.cc:2272] T 9f8b322a475e40fcbca3445b6b0dcf26 P 96a7b6a7b75246d381f5d4a71ea716d9 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:41.840437 10552 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:41.861862 10552 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:42.096028 10553 consensus_peers.cc:597] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:42.213616 10553 consensus_peers.cc:597] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.236877 11177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111) [suppressed 478 similar messages]
W20260709 12:13:42.261314 10553 consensus_peers.cc:597] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:42.361240 11177 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44681, user_credentials={real_user=slave}} blocked reactor thread for 49039.5us
W20260709 12:13:42.436646 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.454943 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.515014 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:42.576196 18856 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.577008 18856 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.580188 18856 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:42.589766 11177 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:42.600133 11178 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.600708 11178 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:42.605734 10553 consensus_peers.cc:597] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:42.606685 18848 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:42.666198 18855 raft_consensus.cc:493] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.667416 18855 raft_consensus.cc:515] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } }
I20260709 12:13:42.670403 18855 leader_election.cc:290] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:42.672144 18848 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.672880 18848 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:42.679260 10552 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:42.684319 11177 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.686316 18848 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:42.708369 18856 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.709252 18856 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.712061 18856 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:42.723217 11178 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.724081 11178 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:42.728950 18841 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:42.736199 18848 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.736937 18848 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.766479 18848 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:42.767598 11177 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.779146 18843 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.779897 18843 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.783839 18856 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.784550 18856 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.787449 18843 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:42.789746 18856 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:42.791953 10552 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:42.793112 10553 leader_election.cc:336] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.793642 10553 leader_election.cc:304] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:42.795101 18843 raft_consensus.cc:2749] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:42.796315 11178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42153, user_credentials={real_user=slave}} blocked reactor thread for 52042.8us
W20260709 12:13:42.802520 11178 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.803136 11178 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:42.812511 18841 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:42.821280 10553 consensus_peers.cc:597] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.845896 10553 consensus_peers.cc:597] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.865811 11177 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:42.866449 11177 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.870261 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:42.874033 11178 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.876686 18843 raft_consensus.cc:493] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.877286 18843 raft_consensus.cc:515] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:42.879952 18843 leader_election.cc:290] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:42.874594 11178 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:42.880913 18841 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:42.882090 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.885154 11178 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:42.885821 10552 leader_election.cc:336] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.892969 18841 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:42.893800 18843 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.894439 18843 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.895807 18855 raft_consensus.cc:493] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.896416 18855 raft_consensus.cc:515] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:42.897518 18843 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:42.898495 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:42.903491 18855 leader_election.cc:290] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:42.904237 10552 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:42.889292 10553 consensus_peers.cc:597] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.908668 10553 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:42.910091 10552 leader_election.cc:336] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.917058 10553 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:42.923154 18843 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:13:42.922736 10553 consensus_peers.cc:597] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:42.927389 10553 consensus_peers.cc:597] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:13:42.929648 18841 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.933826 18841 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.936386 18841 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:42.940747 18856 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.941375 18856 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:42.943768 18856 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:42.954385 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:42.964023 18843 raft_consensus.cc:493] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.964660 18843 raft_consensus.cc:515] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:42.966710 18843 leader_election.cc:290] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:42.967204 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:13:42.970086 11177 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.973119 18855 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:42.976243 11177 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:42.976362 18855 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:42.972658 10552 leader_election.cc:336] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.979296 18841 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:42.981876 11177 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:42.982111 18855 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:42.982622 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
W20260709 12:13:42.984388 10553 leader_election.cc:336] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.990296 10553 leader_election.cc:304] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:42.990540 10552 leader_election.cc:336] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:42.992175 11178 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:42.995036 18843 raft_consensus.cc:2749] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:13:42.998289 11178 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.001889 18841 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:43.014879 18841 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.015487 18841 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.017588 18841 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:43.023430 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:43.024115 10578 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 19.
I20260709 12:13:43.027462 18843 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.028053 18843 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.030102 18843 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:43.032161 10553 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.032699 10553 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.033988 18843 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:43.034708 18855 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.035318 18855 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.037575 18855 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
I20260709 12:13:43.038043 18841 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:43.038273 10552 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.038597 18841 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.041263 18841 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:43.042268 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
W20260709 12:13:43.042889 10553 leader_election.cc:336] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:43.044013 10552 leader_election.cc:336] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.046031 10553 leader_election.cc:304] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.047555 18855 raft_consensus.cc:2749] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:13:43.048169 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:43.048717 18841 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:43.049918 18841 raft_consensus.cc:493] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.050318 18841 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:43.059063 10553 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:43.064337 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "448b86aa128243e48bed5f471ada874f" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:43.065968 10578 raft_consensus.cc:2468] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 25.
I20260709 12:13:43.072448 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:43.074357 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:43.075055 11220 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 19.
W20260709 12:13:43.078094 11177 leader_election.cc:336] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.107566 18843 raft_consensus.cc:493] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.108209 18843 raft_consensus.cc:515] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:43.110265 18843 leader_election.cc:290] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:43.116386 10552 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:43.121057 11178 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.126282 18856 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.126956 18856 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.129637 18856 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:43.133968 18848 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.134634 18848 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.137046 18848 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:43.137573 18848 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.138111 18848 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.143261 18848 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:43.145645 18841 raft_consensus.cc:515] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.148706 10553 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 52bae2d2caf94d97b1049ecd4f82a2df, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:43.153257 11177 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.161115 11177 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.161942 18857 raft_consensus.cc:2749] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:43.165645 18848 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:43.163040 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:43.176617 18857 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:43.177057 18857 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.177428 18857 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:43.182888 18843 raft_consensus.cc:493] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.183501 18843 raft_consensus.cc:515] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
W20260709 12:13:43.191272 10553 leader_election.cc:336] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.191901 10553 leader_election.cc:304] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:43.192826 10552 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:43.172751 11178 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.194949 18855 raft_consensus.cc:2749] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:43.195832 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" is_pre_election: true
I20260709 12:13:43.195873 18843 leader_election.cc:290] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:43.196553 11220 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 14.
I20260709 12:13:43.206373 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:43.207190 10578 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
I20260709 12:13:43.211042 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:43.211692 10578 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 14.
I20260709 12:13:43.214087 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "955c0a21e1e04f51bbbc7c06b5d611f5" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8" is_pre_election: true
I20260709 12:13:43.214811 10578 raft_consensus.cc:2468] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:43.223266 18841 leader_election.cc:290] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:43.189289 18857 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.248950 18855 raft_consensus.cc:493] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.249647 18855 raft_consensus.cc:515] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:43.252599 18848 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.253307 18848 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.255709 18848 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:43.256853 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:43.257648 18855 leader_election.cc:290] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:43.261272 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:43.261850 18848 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:13:43.265196 10552 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.265388 18848 raft_consensus.cc:493] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.265940 18848 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:43.263796  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
I20260709 12:13:43.259912 18857 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68:40987)
W20260709 12:13:43.269313 11178 leader_election.cc:336] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:43.269475 10552 leader_election.cc:336] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.271283 18856 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:13:43.271719 18856 raft_consensus.cc:493] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting leader election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.272094 18856 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:43.272202 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4cd5d330f694dacb199bd61266c8f5a" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:43.272953 10578 raft_consensus.cc:3060] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:43.271271 18855 raft_consensus.cc:493] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.275669 18843 raft_consensus.cc:493] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:43.282086 18843 raft_consensus.cc:515] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:43.282066 18855 raft_consensus.cc:515] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } }
I20260709 12:13:43.283242 10578 raft_consensus.cc:2468] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 20.
I20260709 12:13:43.284451 18843 leader_election.cc:290] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:43.286094 18855 leader_election.cc:290] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:43.294706 18856 raft_consensus.cc:515] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:43.297016 10553 consensus_peers.cc:597] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:43.299777 18856 leader_election.cc:290] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:43.308439 10552 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:43.309011 10552 leader_election.cc:336] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.314567 10578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9042bec741aa476587972e688997da43" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:43.315372 10578 raft_consensus.cc:3060] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:13:43.319689 11177 leader_election.cc:336] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:43.320544 11177 leader_election.cc:336] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.324354 11220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "945dd7b821a5493f8a9d6ca251795da9" candidate_uuid: "33d77e3063684586a737bccf358786d8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "52bae2d2caf94d97b1049ecd4f82a2df"
I20260709 12:13:43.325069 11220 raft_consensus.cc:3060] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:43.325855 10553 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:43.305164 18848 raft_consensus.cc:515] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.337314 11220 raft_consensus.cc:2468] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d77e3063684586a737bccf358786d8 in term 20.
I20260709 12:13:43.341866 18848 leader_election.cc:290] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Requested vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:43.345187 10553 consensus_peers.cc:597] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:43.321357 11177 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.356262 11177 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:43.356256 10553 leader_election.cc:336] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.358256 18848 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:43.361227 11177 leader_election.cc:336] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.357790 10553 leader_election.cc:304] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.364405 18843 raft_consensus.cc:2749] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:43.365777 10553 leader_election.cc:336] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.366358 10553 leader_election.cc:304] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.365854 11177 leader_election.cc:304] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:43.368422 18848 raft_consensus.cc:2804] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:43.369588 18843 raft_consensus.cc:2749] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:43.372570 18848 raft_consensus.cc:697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:43.373847 18848 consensus_queue.cc:237] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.382002 10578 raft_consensus.cc:2468] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 15.
I20260709 12:13:43.388288 10585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfb1df1b6ba3414899a137b410511b26" candidate_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "33d77e3063684586a737bccf358786d8"
I20260709 12:13:43.390177 10585 raft_consensus.cc:3060] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:43.392271 10553 leader_election.cc:336] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.392836 10553 leader_election.cc:304] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.398753 18857 raft_consensus.cc:2764] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:13:43.401734 18857 raft_consensus.cc:2749] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:43.402971 10553 leader_election.cc:336] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.403540 10553 leader_election.cc:304] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.404413 18857 raft_consensus.cc:2749] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:43.405711 10553 leader_election.cc:336] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:43.406193 10553 leader_election.cc:304] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33d77e3063684586a737bccf358786d8; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:43.401826 10585 raft_consensus.cc:2468] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52bae2d2caf94d97b1049ecd4f82a2df in term 21.
I20260709 12:13:43.410465 18857 raft_consensus.cc:2749] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:43.412056 10553 consensus_peers.cc:597] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:43.418779 11177 leader_election.cc:336] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.419965 11177 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
W20260709 12:13:43.427592 11177 leader_election.cc:336] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.433079 18848 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:43.442854 10336 catalog_manager.cc:5697] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 20 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:43.445957 11177 leader_election.cc:304] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:43.454734 18848 raft_consensus.cc:2804] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:13:43.455575 18848 raft_consensus.cc:697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:43.456939 18848 consensus_queue.cc:237] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.462656  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
I20260709 12:13:43.463500 11177 leader_election.cc:304] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 
I20260709 12:13:43.465621 18841 raft_consensus.cc:2804] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:13:43.466840 11177 leader_election.cc:336] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:43.479317 18841 raft_consensus.cc:697] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: Becoming Leader. State: Replica: 52bae2d2caf94d97b1049ecd4f82a2df, State: Running, Role: LEADER
I20260709 12:13:43.480496 18841 consensus_queue.cc:237] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:43.484299 10553 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d77e3063684586a737bccf358786d8, 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:43.486931 18843 raft_consensus.cc:2804] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:43.487421 18843 raft_consensus.cc:697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Becoming Leader. State: Replica: 33d77e3063684586a737bccf358786d8, State: Running, Role: LEADER
I20260709 12:13:43.488382 18843 consensus_queue.cc:237] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:43.531745 10553 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:40987, user_credentials={real_user=slave}} blocked reactor thread for 47565.3us
W20260709 12:13:43.536741 11178 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:13:43.541622 10553 consensus_peers.cc:597] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:13:43.544601 10328 catalog_manager.cc:5697] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 reported cstate change: term changed from 19 to 20, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 33d77e3063684586a737bccf358786d8 (127.9.11.67). New cstate: current_term: 20 leader_uuid: "33d77e3063684586a737bccf358786d8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:43.558386 10553 consensus_peers.cc:597] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:13:43.604990 10336 catalog_manager.cc:5697] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 14 to 15, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 15 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:43.607095 10336 catalog_manager.cc:5697] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df reported cstate change: term changed from 20 to 21, leader changed from 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66) to 52bae2d2caf94d97b1049ecd4f82a2df (127.9.11.68). New cstate: current_term: 21 leader_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:43.624167 11177 consensus_peers.cc:597] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:43.647168 11178 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:43.661867 11177 consensus_peers.cc:597] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.667663  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:13:43.671578  9261 tablet_replica.cc:333] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.672844  9261 raft_consensus.cc:2243] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.673686  9261 raft_consensus.cc:2272] T 73ade8620f2544acbb1e92193cf221b7 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.679188  9261 tablet_replica.cc:333] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.680287  9261 raft_consensus.cc:2243] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.680966  9261 raft_consensus.cc:2272] T e81a69eea26d461e9662b9b2aa4151cd P 33d77e3063684586a737bccf358786d8 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.684319  9261 tablet_replica.cc:333] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.685418  9261 raft_consensus.cc:2243] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.686861  9261 pending_rounds.cc:70] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Trying to abort 1 pending ops.
I20260709 12:13:43.687911  9261 pending_rounds.cc:77] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8: Aborting op as it isn't in flight: id { term: 20 index: 15 } timestamp: 7305622419501715456 op_type: NO_OP noop_request { }
I20260709 12:13:43.688519  9261 raft_consensus.cc:2889] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:13:43.688933  9261 raft_consensus.cc:2272] T 945dd7b821a5493f8a9d6ca251795da9 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.693481  9261 tablet_replica.cc:333] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.694542  9261 raft_consensus.cc:2243] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.695665  9261 raft_consensus.cc:2272] T 448b86aa128243e48bed5f471ada874f P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.698782  9261 tablet_replica.cc:333] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.699786  9261 raft_consensus.cc:2243] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.700564  9261 raft_consensus.cc:2272] T 38ead18e1a814c8b958f80396faa3170 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:43.702960 11177 consensus_peers.cc:597] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.705590  9261 tablet_replica.cc:333] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.706684  9261 raft_consensus.cc:2243] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.708211  9261 raft_consensus.cc:2272] T b4d2e9ee3d824817989d1fa5abcd9d67 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.711588  9261 tablet_replica.cc:333] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.712453  9261 raft_consensus.cc:2243] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.713030  9261 raft_consensus.cc:2272] T 7919194a8a8542559be104753f043851 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.716085  9261 tablet_replica.cc:333] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.716984  9261 raft_consensus.cc:2243] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.717852  9261 raft_consensus.cc:2272] T 58ffd928f77840c18d76b0b748c8ac26 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.721165  9261 tablet_replica.cc:333] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.721884  9261 raft_consensus.cc:2243] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.722904  9261 raft_consensus.cc:2272] T 1ecefe266eea48c4af129b425f391fe9 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.725512  9261 tablet_replica.cc:333] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.726215  9261 raft_consensus.cc:2243] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.727293  9261 raft_consensus.cc:2272] T f94d710c36f04a9b8685258a7ff9d8a0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.729687  9261 tablet_replica.cc:333] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.730610  9261 raft_consensus.cc:2243] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.732476  9261 raft_consensus.cc:2272] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:43.734555 11177 consensus_peers.cc:597] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.735672  9261 tablet_replica.cc:333] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.736460  9261 raft_consensus.cc:2243] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.737838  9261 raft_consensus.cc:2272] T a80f9fe1a0f840c4a6b9984e5fc64eed P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.741010  9261 tablet_replica.cc:333] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.741981  9261 raft_consensus.cc:2243] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.743306  9261 raft_consensus.cc:2272] T 2e3622c19ce24dc4a8573de044a7656b P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.746331  9261 tablet_replica.cc:333] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.747148  9261 raft_consensus.cc:2243] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.748092  9261 raft_consensus.cc:2272] T deab11eaf55b4f3ebe48926a1e2afa38 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.750360  9261 tablet_replica.cc:333] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.751117  9261 raft_consensus.cc:2243] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.751705  9261 raft_consensus.cc:2272] T 5613e87b5a8c4948ae15672ea77f31cf P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.754292  9261 tablet_replica.cc:333] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.755048  9261 raft_consensus.cc:2243] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.756306  9261 raft_consensus.cc:2272] T cc34ac39d2184bffb94d60fe15003395 P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.759275  9261 tablet_replica.cc:333] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.760272  9261 raft_consensus.cc:2243] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.761552  9261 raft_consensus.cc:2272] T 859a2c4bf0aa4585ad9a84c0c4fbdb2d P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.764743  9261 tablet_replica.cc:333] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.765614  9261 raft_consensus.cc:2243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.766283  9261 raft_consensus.cc:2272] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.769366  9261 tablet_replica.cc:333] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.771082  9261 raft_consensus.cc:2243] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:13:43.772156 10552 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:43.772421  9261 raft_consensus.cc:2272] T a0bb77704fc9403481da02083315dbb1 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.775718  9261 tablet_replica.cc:333] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.776893  9261 raft_consensus.cc:2243] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.777532  9261 raft_consensus.cc:2272] T e4cd5d330f694dacb199bd61266c8f5a P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.780543  9261 tablet_replica.cc:333] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
W20260709 12:13:43.780434 11177 consensus_peers.cc:597] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.781889  9261 raft_consensus.cc:2243] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.783541  9261 raft_consensus.cc:2272] T 9a78d11ad8304de6b3c842374e1a2e09 P 33d77e3063684586a737bccf358786d8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.786787  9261 tablet_replica.cc:333] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.787686  9261 raft_consensus.cc:2243] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.789362  9261 raft_consensus.cc:2272] T c545de01b7fb4c89946871c1d522006f P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.792164  9261 tablet_replica.cc:333] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.793025  9261 raft_consensus.cc:2243] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.793632  9261 raft_consensus.cc:2272] T bea092b9128d46b6a42f4fe9aa23c739 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.797195  9261 tablet_replica.cc:333] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.798072  9261 raft_consensus.cc:2243] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:13:43.798069 10553 consensus_peers.cc:597] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:13:43.799301  9261 raft_consensus.cc:2272] T 361eb2421d9548008ac1c2e32ba9ec14 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.802390  9261 tablet_replica.cc:333] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.804023  9261 raft_consensus.cc:2243] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.805452  9261 raft_consensus.cc:2272] T cfb1df1b6ba3414899a137b410511b26 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.808609  9261 tablet_replica.cc:333] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.809481  9261 raft_consensus.cc:2243] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.810294  9261 raft_consensus.cc:2272] T 60f72c0bed7c45e484efbed50d2c364a P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.813277  9261 tablet_replica.cc:333] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.814128  9261 raft_consensus.cc:2243] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.815578  9261 raft_consensus.cc:2272] T 6056d1f6a88d41d383e05a2c04508741 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.818332  9261 tablet_replica.cc:333] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.819178  9261 raft_consensus.cc:2243] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.819829  9261 raft_consensus.cc:2272] T 06d2724d97aa43298ebaf9e746580fd0 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.822672  9261 tablet_replica.cc:333] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.823472  9261 raft_consensus.cc:2243] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.824162  9261 raft_consensus.cc:2272] T 31117e61be004c52a161168a559023fd P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.827060  9261 tablet_replica.cc:333] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.828510  9261 raft_consensus.cc:2243] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.829627  9261 raft_consensus.cc:2272] T dc98cd5edc9240fba53978ed59191cbd P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.832414  9261 tablet_replica.cc:333] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.833429  9261 raft_consensus.cc:2243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.834438  9261 raft_consensus.cc:2272] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.838119  9261 tablet_replica.cc:333] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.838964  9261 raft_consensus.cc:2243] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.842098  9261 raft_consensus.cc:2272] T 9bfb73cee8704df696d2eed4d27003bb P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.845297  9261 tablet_replica.cc:333] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
W20260709 12:13:43.846009 10552 consensus_peers.cc:597] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:43.846124  9261 raft_consensus.cc:2243] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.847697  9261 raft_consensus.cc:2272] T f5d55f0ab8c4445abbbca950ea76dca1 P 33d77e3063684586a737bccf358786d8 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.850824  9261 tablet_replica.cc:333] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.851699  9261 raft_consensus.cc:2243] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.852342  9261 raft_consensus.cc:2272] T 05e004719a9442e2b9b045e4a6df5426 P 33d77e3063684586a737bccf358786d8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.855024  9261 tablet_replica.cc:333] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.855865  9261 raft_consensus.cc:2243] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 17 LEADER]: Raft consensus shutting down.
W20260709 12:13:43.856715 11177 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:43.857712 11177 consensus_peers.cc:597] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.857766  9261 raft_consensus.cc:2272] T 4baf326f866a4a17b411837d7902a0c7 P 33d77e3063684586a737bccf358786d8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.862900  9261 tablet_replica.cc:333] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.864817  9261 raft_consensus.cc:2243] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.865546  9261 raft_consensus.cc:2272] T 63843cf9ccd4442cb52e3988597cabd8 P 33d77e3063684586a737bccf358786d8 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:43.867609 11177 consensus_peers.cc:597] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:43.871392 11177 consensus_peers.cc:597] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.869323  9261 tablet_replica.cc:333] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.875208  9261 raft_consensus.cc:2243] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.876119  9261 raft_consensus.cc:2272] T c882875c377649acb233035d47b8132e P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.879554  9261 tablet_replica.cc:333] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.880683  9261 raft_consensus.cc:2243] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.881346  9261 raft_consensus.cc:2272] T 9042bec741aa476587972e688997da43 P 33d77e3063684586a737bccf358786d8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.884419  9261 tablet_replica.cc:333] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.885332  9261 raft_consensus.cc:2243] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.886641  9261 raft_consensus.cc:2272] T 147a37ae746d4efb84fdc2f6d4d5981e P 33d77e3063684586a737bccf358786d8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.889530  9261 tablet_replica.cc:333] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.890421  9261 raft_consensus.cc:2243] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.891122  9261 raft_consensus.cc:2272] T 4689962670ee421e8993be8a9ba7a657 P 33d77e3063684586a737bccf358786d8 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.894281  9261 tablet_replica.cc:333] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.895196  9261 raft_consensus.cc:2243] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.896502  9261 raft_consensus.cc:2272] T 977ae790c1464057bdfdea6a1466fdc3 P 33d77e3063684586a737bccf358786d8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.899855  9261 tablet_replica.cc:333] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.900689  9261 raft_consensus.cc:2243] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.902057  9261 raft_consensus.cc:2272] T c7366ca68fce42f58f5e546ae5cad3ca P 33d77e3063684586a737bccf358786d8 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:43.904657 11177 consensus_peers.cc:597] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.905299  9261 tablet_replica.cc:333] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
W20260709 12:13:43.905366 11177 consensus_peers.cc:597] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:43.906361  9261 raft_consensus.cc:2243] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.907624  9261 raft_consensus.cc:2272] T b117a8eaf2494d0ea2317e5895cac532 P 33d77e3063684586a737bccf358786d8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.910528  9261 tablet_replica.cc:333] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.911440  9261 raft_consensus.cc:2243] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:43.912218  9261 raft_consensus.cc:2272] T c5a50eeaf1ac462883e185eddec889f6 P 33d77e3063684586a737bccf358786d8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.915550  9261 tablet_replica.cc:333] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.916328  9261 raft_consensus.cc:2243] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.917588  9261 raft_consensus.cc:2272] T 773a46c59e194295b8f5f7c5b6d58216 P 33d77e3063684586a737bccf358786d8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.920838  9261 tablet_replica.cc:333] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.921563  9261 raft_consensus.cc:2243] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:43.921693 11177 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:43.922338  9261 raft_consensus.cc:2272] T d62573c8e5a64ece95bc46d5fa18185c P 33d77e3063684586a737bccf358786d8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:43.925570  9261 tablet_replica.cc:333] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8: stopping tablet replica
I20260709 12:13:43.926282  9261 raft_consensus.cc:2243] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:43.927596  9261 raft_consensus.cc:2272] T 80de1e7bce9c4d34b0583315cafbc371 P 33d77e3063684586a737bccf358786d8 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:43.929514 11177 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:43.941048 11177 consensus_peers.cc:597] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:43.997936 11177 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:44.100577 11177 consensus_peers.cc:597] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:44.104809 11177 consensus_peers.cc:597] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Couldn't send request to peer 33d77e3063684586a737bccf358786d8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:44.250216 11177 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:44.265359 11178 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:13:44.265996 11178 consensus_peers.cc:597] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:13:44.284567 11177 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:44.285248 11177 consensus_peers.cc:597] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:44.285636 11177 consensus_peers.cc:597] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:44.295076 11177 consensus_peers.cc:597] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:13:44.295581 11177 consensus_peers.cc:597] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:13:44.375455 11177 consensus_peers.cc:597] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:44.432376 18876 raft_consensus.cc:493] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:44.433740 18876 raft_consensus.cc:515] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:44.442466 11178 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:44.444732 11177 consensus_peers.cc:597] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:44.445515 18876 leader_election.cc:290] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:44.454604 11177 leader_election.cc:336] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:44.455166 11177 leader_election.cc:304] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:44.456548 18876 raft_consensus.cc:2749] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:44.480360 18877 raft_consensus.cc:493] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:44.481349 18877 raft_consensus.cc:515] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:44.484038 18877 leader_election.cc:290] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:44.495507 11178 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:44.516279 11177 leader_election.cc:336] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:44.516855 11177 leader_election.cc:304] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:44.517666 18876 raft_consensus.cc:2749] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:44.518821 11177 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Couldn't send request to peer 96a7b6a7b75246d381f5d4a71ea716d9. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:13:44.633641 18876 raft_consensus.cc:493] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:44.634507 18876 raft_consensus.cc:515] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:44.643141 11178 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:44.644773 18876 leader_election.cc:290] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:44.650480 11177 leader_election.cc:336] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:44.651042 11177 leader_election.cc:304] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:44.652446 18876 raft_consensus.cc:2749] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:44.824029 18848 raft_consensus.cc:493] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:44.825009 18848 raft_consensus.cc:515] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:44.829492 18848 leader_election.cc:290] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:44.835222 11178 leader_election.cc:336] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:44.872678 18876 raft_consensus.cc:493] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:44.887166 18876 raft_consensus.cc:515] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:44.889799 18876 leader_election.cc:290] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:44.892309 11177 leader_election.cc:336] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:44.892899 11177 leader_election.cc:304] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:44.893915 18876 raft_consensus.cc:2749] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:13:44.896081 11177 leader_election.cc:336] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:44.911758 11178 consensus_peers.cc:597] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:13:44.912935 18876 raft_consensus.cc:493] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:44.913870 18876 raft_consensus.cc:515] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:44.916492 18876 leader_election.cc:290] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:44.923076 11178 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:44.932528 11177 leader_election.cc:336] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:44.935544 11177 leader_election.cc:336] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:44.936151 11177 leader_election.cc:304] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:44.937170 18876 raft_consensus.cc:2749] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:44.945245 18876 raft_consensus.cc:493] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:44.945906 18876 raft_consensus.cc:515] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:44.955552 11178 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:44.950742 18876 leader_election.cc:290] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:44.958719 11177 leader_election.cc:336] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:44.960911 11177 leader_election.cc:304] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:44.961745 18876 raft_consensus.cc:2749] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:44.976456 11177 leader_election.cc:336] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:44.977124 11177 leader_election.cc:304] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:44.978046 18876 raft_consensus.cc:2749] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:45.009671 11178 consensus_peers.cc:597] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:13:45.012229 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08001738c0 after lost signal to thread 9264
W20260709 12:13:45.014519 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08003dc420 after lost signal to thread 10370
W20260709 12:13:45.015781 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08001d34a0 after lost signal to thread 10611
W20260709 12:13:45.021159 10367 debug-util.cc:398] Leaking SignalData structure 0x7b0800222fe0 after lost signal to thread 11248
I20260709 12:13:45.043344 18876 raft_consensus.cc:493] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.044368 18876 raft_consensus.cc:515] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.049691 18848 raft_consensus.cc:493] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.050473 18848 raft_consensus.cc:515] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.052652 18848 leader_election.cc:290] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:45.053186 18848 raft_consensus.cc:493] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.053823 18848 raft_consensus.cc:515] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.055160 18877 raft_consensus.cc:493] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.055856 18877 raft_consensus.cc:515] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.058218 18877 leader_election.cc:290] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.062932 11178 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.064867 18876 leader_election.cc:290] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.073398 11178 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.076151 18848 leader_election.cc:290] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:45.075380 18876 raft_consensus.cc:493] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
W20260709 12:13:45.076202 11178 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.080904 18876 raft_consensus.cc:515] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.084363 18881 raft_consensus.cc:493] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:45.074817  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:13:45.086679 18876 leader_election.cc:290] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:45.091997 18848 raft_consensus.cc:493] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
W20260709 12:13:45.091588 11178 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.093091 18848 raft_consensus.cc:515] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } attrs { replace: true } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.096958 18848 leader_election.cc:290] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:45.099640 18881 raft_consensus.cc:515] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:45.108140 11177 leader_election.cc:336] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.112041 11177 leader_election.cc:304] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.111338 18881 leader_election.cc:290] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
I20260709 12:13:45.113200 18876 raft_consensus.cc:2749] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:13:45.109956 11178 consensus_peers.cc:597] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:13:45.114029 11178 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:45.117523 11178 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:45.118139 11177 leader_election.cc:336] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.119549 11177 leader_election.cc:304] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.120599 18876 raft_consensus.cc:2749] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:13:45.120863 11177 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.121508 11177 leader_election.cc:304] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.122509 18876 raft_consensus.cc:2749] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:13:45.123270 11177 leader_election.cc:336] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.123893 11177 leader_election.cc:304] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.124784 18876 raft_consensus.cc:2749] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:13:45.126268 11177 leader_election.cc:336] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:45.127094 11178 leader_election.cc:336] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.127737 18876 raft_consensus.cc:493] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.128325 18876 raft_consensus.cc:515] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.130650 18876 leader_election.cc:290] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.132223 11177 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:45.132834 11177 leader_election.cc:304] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.133608 18881 raft_consensus.cc:2749] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:45.135437 11177 leader_election.cc:336] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.135984 11177 leader_election.cc:304] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.137256 18876 raft_consensus.cc:2749] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:45.137471 11177 leader_election.cc:336] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
W20260709 12:13:45.137247 11178 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:45.143065 11177 leader_election.cc:336] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.143591 11177 leader_election.cc:304] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.144534 18876 raft_consensus.cc:2749] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:13:45.148674 11177 leader_election.cc:336] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:45.149466 11177 leader_election.cc:304] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.150555 18876 raft_consensus.cc:2749] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:45.176753 18876 raft_consensus.cc:493] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.177765 18876 raft_consensus.cc:515] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.180351 18876 leader_election.cc:290] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:45.185974 18876 raft_consensus.cc:493] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:45.186707 18876 raft_consensus.cc:515] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.190794 18876 leader_election.cc:290] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681)
W20260709 12:13:45.198171 11178 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
W20260709 12:13:45.202737 11177 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.203794 18876 raft_consensus.cc:493] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.204622 18876 raft_consensus.cc:515] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.206945 18876 leader_election.cc:290] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
I20260709 12:13:45.203967 18881 raft_consensus.cc:493] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.208784 18881 raft_consensus.cc:515] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:45.209175 11177 leader_election.cc:336] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:45.209661 11177 leader_election.cc:304] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:45.213950 18876 raft_consensus.cc:2749] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:45.215444 18881 leader_election.cc:290] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.218430 11178 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.220227 18881 raft_consensus.cc:493] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 33d77e3063684586a737bccf358786d8)
I20260709 12:13:45.221096 18881 raft_consensus.cc:515] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.223948 18881 leader_election.cc:290] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.227957 11177 leader_election.cc:336] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:45.228487 11177 leader_election.cc:304] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.230432 18881 raft_consensus.cc:2749] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:45.231186 11177 leader_election.cc:336] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.235226 11177 leader_election.cc:304] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.236068 18881 raft_consensus.cc:2749] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:45.243408 11177 leader_election.cc:336] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:45.244068 11177 leader_election.cc:336] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:45.244760 11177 leader_election.cc:336] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.245167 11177 leader_election.cc:304] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
W20260709 12:13:45.246170 11177 leader_election.cc:336] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.246343 18881 raft_consensus.cc:2749] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:45.246610 11177 leader_election.cc:304] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:45.249614 18876 raft_consensus.cc:2749] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:45.290534 11178 consensus_peers.cc:597] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:13:45.363984 18876 raft_consensus.cc:493] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 96a7b6a7b75246d381f5d4a71ea716d9)
I20260709 12:13:45.364876 18876 raft_consensus.cc:515] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "99d42e7288f145f1bd5af212fd062995" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42153 } } peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } attrs { replace: true } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
W20260709 12:13:45.376441 11178 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.378417 18876 leader_election.cc:290] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153), 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.379267 11177 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
I20260709 12:13:45.379781 11177 leader_election.cc:304] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 96a7b6a7b75246d381f5d4a71ea716d9, 99d42e7288f145f1bd5af212fd062995
I20260709 12:13:45.380565 18876 raft_consensus.cc:2749] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:13:45.385349 11177 leader_election.cc:336] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.411803 18876 raft_consensus.cc:493] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.412712 18876 raft_consensus.cc:515] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "96a7b6a7b75246d381f5d4a71ea716d9" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44681 } } peers { permanent_uuid: "33d77e3063684586a737bccf358786d8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 36727 } } peers { permanent_uuid: "52bae2d2caf94d97b1049ecd4f82a2df" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 40987 } attrs { promote: false } }
I20260709 12:13:45.415831 18876 leader_election.cc:290] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681), 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727)
W20260709 12:13:45.424091 11178 consensus_peers.cc:597] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:13:45.441445 11177 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 96a7b6a7b75246d381f5d4a71ea716d9 (127.9.11.66:44681): Network error: Client connection negotiation failed: client connection to 127.9.11.66:44681: connect: Connection refused (error 111)
W20260709 12:13:45.442519 11177 leader_election.cc:336] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 33d77e3063684586a737bccf358786d8 (127.9.11.67:36727): Network error: Client connection negotiation failed: client connection to 127.9.11.67:36727: connect: Connection refused (error 111)
I20260709 12:13:45.443022 11177 leader_election.cc:304] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 52bae2d2caf94d97b1049ecd4f82a2df; no voters: 33d77e3063684586a737bccf358786d8, 96a7b6a7b75246d381f5d4a71ea716d9
I20260709 12:13:45.443888 18876 raft_consensus.cc:2749] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:45.503997  9261 tablet_server.cc:179] TabletServer@127.9.11.68:0 shutting down...
I20260709 12:13:45.653224  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:13:45.654565  9261 tablet_replica.cc:333] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.656365  9261 raft_consensus.cc:2243] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.660492  9261 raft_consensus.cc:2272] T 773a46c59e194295b8f5f7c5b6d58216 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.665321  9261 tablet_replica.cc:333] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.666615  9261 raft_consensus.cc:2243] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.667680  9261 raft_consensus.cc:2272] T 448b86aa128243e48bed5f471ada874f P 52bae2d2caf94d97b1049ecd4f82a2df [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.680320  9261 tablet_replica.cc:333] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.681785  9261 raft_consensus.cc:2243] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.683226  9261 raft_consensus.cc:2272] T 90f71c8f44dc4ef79c01a6c9cd76cc61 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.687786  9261 tablet_replica.cc:333] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.688932  9261 raft_consensus.cc:2243] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.690673  9261 raft_consensus.cc:2272] T 945dd7b821a5493f8a9d6ca251795da9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.706162  9261 tablet_replica.cc:333] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.719816  9261 raft_consensus.cc:2243] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.721689  9261 raft_consensus.cc:2272] T c61f4bfa2db8497dbf7e0446e2ab707d P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.726559  9261 tablet_replica.cc:333] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.728109  9261 raft_consensus.cc:2243] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.730154  9261 raft_consensus.cc:2272] T 1ecefe266eea48c4af129b425f391fe9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.734253  9261 tablet_replica.cc:333] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
W20260709 12:13:45.734205 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:13:45.735939  9261 raft_consensus.cc:2243] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.738057  9261 raft_consensus.cc:2272] T 7919194a8a8542559be104753f043851 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.744020  9261 tablet_replica.cc:333] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.745314  9261 raft_consensus.cc:2243] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.747690  9261 raft_consensus.cc:2272] T 4fa6e3eaefd14aff89099fd64eeb1a1d P 52bae2d2caf94d97b1049ecd4f82a2df [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.758975  9261 tablet_replica.cc:333] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.760253  9261 raft_consensus.cc:2243] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.762647  9261 pending_rounds.cc:70] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Trying to abort 1 pending ops.
I20260709 12:13:45.763098  9261 pending_rounds.cc:77] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting op as it isn't in flight: id { term: 21 index: 15 } timestamp: 7305622419385831424 op_type: NO_OP noop_request { }
I20260709 12:13:45.764842  9261 raft_consensus.cc:2889] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:13:45.765326  9261 raft_consensus.cc:2272] T cfb1df1b6ba3414899a137b410511b26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.769254  9261 tablet_replica.cc:333] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.770356  9261 raft_consensus.cc:2243] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.772356  9261 raft_consensus.cc:2272] T 81b70ca03fbe4622a03717efd079d598 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.775851  9261 tablet_replica.cc:333] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.776885  9261 raft_consensus.cc:2243] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.778164  9261 raft_consensus.cc:2272] T deab11eaf55b4f3ebe48926a1e2afa38 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.781455  9261 tablet_replica.cc:333] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.782159  9261 raft_consensus.cc:2243] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.783506  9261 raft_consensus.cc:2272] T 51452865e973445d9e13f232cfb5efb9 P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.786264  9261 tablet_replica.cc:333] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.787410  9261 raft_consensus.cc:2243] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.788296  9261 raft_consensus.cc:2272] T e8daad82d9f346a295fe1cf354383acc P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.791966  9261 tablet_replica.cc:333] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.793812  9261 raft_consensus.cc:2243] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.794818  9261 raft_consensus.cc:2272] T 66985e55da4c4d3e88491e146eaaded2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.797348  9261 tablet_replica.cc:333] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.797956  9261 raft_consensus.cc:2243] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.799207  9261 raft_consensus.cc:2272] T 60f72c0bed7c45e484efbed50d2c364a P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.801600  9261 tablet_replica.cc:333] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.802256  9261 raft_consensus.cc:2243] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.803333  9261 raft_consensus.cc:2272] T b3872158fef542f0b0f902e3a8756feb P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.805981  9261 tablet_replica.cc:333] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.806589  9261 raft_consensus.cc:2243] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.807860  9261 pending_rounds.cc:70] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Trying to abort 1 pending ops.
I20260709 12:13:45.808080  9261 pending_rounds.cc:77] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting op as it isn't in flight: id { term: 15 index: 14 } timestamp: 7305622419296309248 op_type: NO_OP noop_request { }
I20260709 12:13:45.808403  9261 raft_consensus.cc:2889] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:13:45.808713  9261 raft_consensus.cc:2272] T 9042bec741aa476587972e688997da43 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.811232  9261 tablet_replica.cc:333] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.811935  9261 raft_consensus.cc:2243] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.813537  9261 raft_consensus.cc:2272] T 147a37ae746d4efb84fdc2f6d4d5981e P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.818280  9261 tablet_replica.cc:333] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.819341  9261 raft_consensus.cc:2243] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.824236  9261 raft_consensus.cc:2272] T 4689962670ee421e8993be8a9ba7a657 P 52bae2d2caf94d97b1049ecd4f82a2df [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.827687  9261 tablet_replica.cc:333] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.828655  9261 raft_consensus.cc:2243] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.829480  9261 raft_consensus.cc:2272] T c7366ca68fce42f58f5e546ae5cad3ca P 52bae2d2caf94d97b1049ecd4f82a2df [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.832926  9261 tablet_replica.cc:333] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.833791  9261 raft_consensus.cc:2243] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.835299  9261 raft_consensus.cc:2272] T 9f8b322a475e40fcbca3445b6b0dcf26 P 52bae2d2caf94d97b1049ecd4f82a2df [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.838521  9261 tablet_replica.cc:333] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.839468  9261 raft_consensus.cc:2243] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:13:45.839658 11178 consensus_peers.cc:597] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df -> Peer 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): Couldn't send request to peer 99d42e7288f145f1bd5af212fd062995. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:13:45.840209  9261 raft_consensus.cc:2272] T 80de1e7bce9c4d34b0583315cafbc371 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.842542  9261 tablet_replica.cc:333] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.843331  9261 raft_consensus.cc:2243] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.843926  9261 raft_consensus.cc:2272] T 9a78d11ad8304de6b3c842374e1a2e09 P 52bae2d2caf94d97b1049ecd4f82a2df [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.846451  9261 tablet_replica.cc:333] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.847138  9261 raft_consensus.cc:2243] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.848028  9261 raft_consensus.cc:2272] T a082a11e47ba43b2acfaf230875b5e07 P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.850553  9261 tablet_replica.cc:333] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.851220  9261 raft_consensus.cc:2243] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.852640  9261 raft_consensus.cc:2272] T 4642a95b220c4ecea8115de1fa98954e P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.855118  9261 tablet_replica.cc:333] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.855788  9261 raft_consensus.cc:2243] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.856992  9261 raft_consensus.cc:2272] T bea092b9128d46b6a42f4fe9aa23c739 P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.859427  9261 tablet_replica.cc:333] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.860170  9261 raft_consensus.cc:2243] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.861104  9261 raft_consensus.cc:2272] T 4baf326f866a4a17b411837d7902a0c7 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.863979  9261 tablet_replica.cc:333] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.864616  9261 raft_consensus.cc:2243] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.865868  9261 raft_consensus.cc:2272] T 63843cf9ccd4442cb52e3988597cabd8 P 52bae2d2caf94d97b1049ecd4f82a2df [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.868322  9261 tablet_replica.cc:333] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.868930  9261 raft_consensus.cc:2243] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.871337  9261 raft_consensus.cc:2272] T a0bb77704fc9403481da02083315dbb1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.873656  9261 tablet_replica.cc:333] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.874365  9261 raft_consensus.cc:2243] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.877943  9261 raft_consensus.cc:2272] T f8b08b19bdbb4ac7b6bae9eccebb6335 P 52bae2d2caf94d97b1049ecd4f82a2df [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.880829  9261 tablet_replica.cc:333] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.881806  9261 raft_consensus.cc:2243] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.883383  9261 raft_consensus.cc:2272] T 38ead18e1a814c8b958f80396faa3170 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.886307  9261 tablet_replica.cc:333] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.886976  9261 raft_consensus.cc:2243] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.887799  9261 raft_consensus.cc:2272] T 0b474d9b241d4a698702287ee1bbe3b2 P 52bae2d2caf94d97b1049ecd4f82a2df [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.891080  9261 tablet_replica.cc:333] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.891960  9261 raft_consensus.cc:2243] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.893306  9261 raft_consensus.cc:2272] T c545de01b7fb4c89946871c1d522006f P 52bae2d2caf94d97b1049ecd4f82a2df [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.896803  9261 tablet_replica.cc:333] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.897462  9261 raft_consensus.cc:2243] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.898841  9261 pending_rounds.cc:70] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Trying to abort 1 pending ops.
I20260709 12:13:45.899108  9261 pending_rounds.cc:77] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305622419056758784 op_type: NO_OP noop_request { }
I20260709 12:13:45.899469  9261 raft_consensus.cc:2889] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:13:45.899798  9261 raft_consensus.cc:2272] T e4cd5d330f694dacb199bd61266c8f5a P 52bae2d2caf94d97b1049ecd4f82a2df [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.902364  9261 tablet_replica.cc:333] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.903024  9261 raft_consensus.cc:2243] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.903646  9261 raft_consensus.cc:2272] T b4d2e9ee3d824817989d1fa5abcd9d67 P 52bae2d2caf94d97b1049ecd4f82a2df [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.906032  9261 tablet_replica.cc:333] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.906648  9261 raft_consensus.cc:2243] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.907441  9261 raft_consensus.cc:2272] T 26d3d17d58be40c8974c348ca146a951 P 52bae2d2caf94d97b1049ecd4f82a2df [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.910385  9261 tablet_replica.cc:333] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.911064  9261 raft_consensus.cc:2243] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.911922  9261 raft_consensus.cc:2272] T 955c0a21e1e04f51bbbc7c06b5d611f5 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.914698  9261 tablet_replica.cc:333] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.915508  9261 raft_consensus.cc:2243] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.916800  9261 raft_consensus.cc:2272] T dc98cd5edc9240fba53978ed59191cbd P 52bae2d2caf94d97b1049ecd4f82a2df [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.923055  9261 tablet_replica.cc:333] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.923713  9261 raft_consensus.cc:2243] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:13:45.924981  9261 raft_consensus.cc:2272] T 06d2724d97aa43298ebaf9e746580fd0 P 52bae2d2caf94d97b1049ecd4f82a2df [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:45.927815  9261 tablet_replica.cc:333] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df: stopping tablet replica
I20260709 12:13:45.928347  9261 raft_consensus.cc:2243] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:45.928879  9261 raft_consensus.cc:2272] T f5d55f0ab8c4445abbbca950ea76dca1 P 52bae2d2caf94d97b1049ecd4f82a2df [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:46.605594  9261 tablet_server.cc:196] TabletServer@127.9.11.68:0 shutdown complete.
I20260709 12:13:46.704490  9261 master.cc:562] Master@127.9.11.126:42677 shutting down...
W20260709 12:13:46.753116 10384 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:13:46.753518 10384 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:13:46.753782 10384 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:13:57.719040 10367 debug-util.cc:398] Leaking SignalData structure 0x7b08000df400 after lost signal to thread 9264
W20260709 12:13:57.719913 10367 debug-util.cc:398] Leaking SignalData structure 0x7b0800223060 after lost signal to thread 10370
W20260709 12:14:02.134759 10323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111) [suppressed 417 similar messages]
W20260709 12:14:02.143065 10323 catalog_manager.cc:4755] TS 99d42e7288f145f1bd5af212fd062995 (127.9.11.65:42153): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 773a46c59e194295b8f5f7c5b6d58216: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42153: connect: Connection refused (error 111)
I20260709 12:14:03.776556  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:03.777892  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:03.778388  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P 92c087afe23c499b9e821ff6f85b7e2a: stopping tablet replica
I20260709 12:14:04.017108  9261 master.cc:584] Master@127.9.11.126:42677 shutdown complete.
I20260709 12:14:04.088655  9261 test_util.cc:182] -----------------------------------------------
I20260709 12:14:04.088956  9261 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598894027500-9261-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (287217 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:14:04.093617  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:34843
I20260709 12:14:04.095082  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:04.102077 18889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.102161 18888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.102969 18891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:04.105384  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:04.111091  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:04.111330  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:04.111531  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599244111494 us; error 0 us; skew 500 ppm
I20260709 12:14:04.112165  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:04.115226  9261 webserver.cc:533] Webserver started at http://127.9.11.126:46155/ using document root <none> and password file <none>
I20260709 12:14:04.115829  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:04.116044  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:04.116359  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:04.117851  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "01467632324a4240b0c52f8513bb99ba"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.123106  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.007s
I20260709 12:14:04.127121 18896 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.128883  9261 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:14:04.129251  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "01467632324a4240b0c52f8513bb99ba"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.129653  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:04.142378  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:04.144068  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:04.193401  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:34843
I20260709 12:14:04.193545 18947 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:34843 every 8 connection(s)
I20260709 12:14:04.198848 18948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:04.214359 18948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba: Bootstrap starting.
I20260709 12:14:04.221107 18948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:04.226912 18948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba: No bootstrap required, opened a new log
I20260709 12:14:04.229859 18948 raft_consensus.cc:359] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01467632324a4240b0c52f8513bb99ba" member_type: VOTER }
I20260709 12:14:04.230418 18948 raft_consensus.cc:385] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:04.230672 18948 raft_consensus.cc:740] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01467632324a4240b0c52f8513bb99ba, State: Initialized, Role: FOLLOWER
I20260709 12:14:04.231446 18948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01467632324a4240b0c52f8513bb99ba" member_type: VOTER }
I20260709 12:14:04.232013 18948 raft_consensus.cc:399] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:14:04.232259 18948 raft_consensus.cc:493] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:14:04.232651 18948 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:04.238857 18948 raft_consensus.cc:515] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01467632324a4240b0c52f8513bb99ba" member_type: VOTER }
I20260709 12:14:04.239609 18948 leader_election.cc:304] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01467632324a4240b0c52f8513bb99ba; no voters: 
I20260709 12:14:04.241091 18948 leader_election.cc:290] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:14:04.241746 18951 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:04.243443 18951 raft_consensus.cc:697] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 1 LEADER]: Becoming Leader. State: Replica: 01467632324a4240b0c52f8513bb99ba, State: Running, Role: LEADER
I20260709 12:14:04.244311 18951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01467632324a4240b0c52f8513bb99ba" member_type: VOTER }
I20260709 12:14:04.244745 18948 sys_catalog.cc:565] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:14:04.252254 18953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01467632324a4240b0c52f8513bb99ba. Latest consensus state: current_term: 1 leader_uuid: "01467632324a4240b0c52f8513bb99ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01467632324a4240b0c52f8513bb99ba" member_type: VOTER } }
I20260709 12:14:04.253089 18953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [sys.catalog]: This master's current role is: LEADER
I20260709 12:14:04.254571 18952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01467632324a4240b0c52f8513bb99ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01467632324a4240b0c52f8513bb99ba" member_type: VOTER } }
I20260709 12:14:04.255542 18952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [sys.catalog]: This master's current role is: LEADER
I20260709 12:14:04.258507 18961 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:14:04.264992 18961 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:14:04.266227  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:14:04.276625 18961 catalog_manager.cc:1383] Generated new cluster ID: fe8651d740b84c0d9e739aaaa99e3a76
I20260709 12:14:04.276980 18961 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:14:04.303516 18961 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:14:04.305068 18961 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:14:04.317579 18961 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba: Generated new TSK 0
I20260709 12:14:04.318291 18961 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:14:04.334467  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:04.342667 18969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.343986 18970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.346140 18972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:04.348626  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:04.352902  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:04.353152  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:04.353385  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599244353362 us; error 0 us; skew 500 ppm
I20260709 12:14:04.354043  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:04.357101  9261 webserver.cc:533] Webserver started at http://127.9.11.65:40003/ using document root <none> and password file <none>
I20260709 12:14:04.357685  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:04.357875  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:04.358166  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:04.359799  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "3fcac1aec8df4c599b4c67550ce347e1"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.365528  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:14:04.369820 18977 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.370762  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:14:04.372021  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "3fcac1aec8df4c599b4c67550ce347e1"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.372385  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:04.408777  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:04.410379  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:04.412598  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:04.433954  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:04.434350  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:14:04.434782  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:04.434965  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.489982  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:43327
I20260709 12:14:04.490226 19040 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:43327 every 8 connection(s)
I20260709 12:14:04.498246  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:04.509374 19045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.510286 19046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.513432 19048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:04.514752  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:04.519179  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:04.519495  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:04.519727  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599244519711 us; error 0 us; skew 500 ppm
I20260709 12:14:04.520579  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:04.523947 19041 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34843
I20260709 12:14:04.524415 19041 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:04.524819  9261 webserver.cc:533] Webserver started at http://127.9.11.66:43937/ using document root <none> and password file <none>
I20260709 12:14:04.525516  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:04.525574 19041 heartbeater.cc:507] Master 127.9.11.126:34843 requested a full tablet report, sending...
I20260709 12:14:04.525750  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:04.526316  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:04.528710  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "666fef4f7e5a4c32813dad8dfade8561"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.529247 18913 ts_manager.cc:194] Registered new tserver with Master: 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:04.532130 18913 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38092
I20260709 12:14:04.536136  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:14:04.540925 19053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.541857  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:14:04.542213  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "666fef4f7e5a4c32813dad8dfade8561"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.542537  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:04.559878  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:04.561695  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:04.563689  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:04.566540  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:04.566769  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.567101  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:04.567272  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.618032  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:34717
I20260709 12:14:04.618172 19115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:34717 every 8 connection(s)
I20260709 12:14:04.625089  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:04.636248 19119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.636729 19120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.639910 19122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:04.641214 19116 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34843
I20260709 12:14:04.641418  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:04.641891 19116 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:04.642867  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:04.643285  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:04.643230 19116 heartbeater.cc:507] Master 127.9.11.126:34843 requested a full tablet report, sending...
I20260709 12:14:04.643692  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599244643667 us; error 0 us; skew 500 ppm
I20260709 12:14:04.644642  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:04.646386 18913 ts_manager.cc:194] Registered new tserver with Master: 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:04.648008  9261 webserver.cc:533] Webserver started at http://127.9.11.67:37387/ using document root <none> and password file <none>
I20260709 12:14:04.648504 18913 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38108
I20260709 12:14:04.648665  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:04.649026  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:04.649406  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:04.651479  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "6cab1cfd695145f992765d39dbe662c7"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.656854  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:14:04.661096 19127 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.662055  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:14:04.662410  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "6cab1cfd695145f992765d39dbe662c7"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.662750  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:04.698807  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:04.700521  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:04.702370  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:04.705295  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:04.705583  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.705860  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:04.706043  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.759850  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:32851
I20260709 12:14:04.760025 19189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:32851 every 8 connection(s)
I20260709 12:14:04.766820  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:04.778080 19194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:04.778718 19193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:04.780084  9261 server_base.cc:1061] running on GCE node
W20260709 12:14:04.780336 19196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:04.781515  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:04.781808  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:04.782042  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599244782020 us; error 0 us; skew 500 ppm
I20260709 12:14:04.782883  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:04.783861 19190 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34843
I20260709 12:14:04.784478 19190 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:04.785629 19190 heartbeater.cc:507] Master 127.9.11.126:34843 requested a full tablet report, sending...
I20260709 12:14:04.786306  9261 webserver.cc:533] Webserver started at http://127.9.11.68:41825/ using document root <none> and password file <none>
I20260709 12:14:04.787180  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:04.787410  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:04.787734  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:04.788465 18913 ts_manager.cc:194] Registered new tserver with Master: 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:04.789337  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-3-root/instance:
uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.790252 18913 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38124
I20260709 12:14:04.795508  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:14:04.799762 19201 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.800638  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:14:04.800998  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-3-root
uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:04.801354  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:04.814050  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:04.815858  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:04.817504  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:04.820050  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:04.820250  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.820564  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:04.820713  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:04.873531  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.68:33373
I20260709 12:14:04.873803 19263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.68:33373 every 8 connection(s)
I20260709 12:14:04.890923 19264 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34843
I20260709 12:14:04.891508 19264 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:04.892540 19264 heartbeater.cc:507] Master 127.9.11.126:34843 requested a full tablet report, sending...
I20260709 12:14:04.894697 18913 ts_manager.cc:194] Registered new tserver with Master: 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:04.895177  9261 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.017660012s
I20260709 12:14:04.896885 18913 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38140
I20260709 12:14:04.931680 18913 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38150:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:14:05.225349 19006 tablet_service.cc:1511] Processing CreateTablet for tablet fbccd6116cc5452ebe3b6f446963bcd2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:14:05.227562 19006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbccd6116cc5452ebe3b6f446963bcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.237639 19005 tablet_service.cc:1511] Processing CreateTablet for tablet 64a4de1a07654315ae7678c1f9e8fcca (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:14:05.240048 19005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a4de1a07654315ae7678c1f9e8fcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.249320 19004 tablet_service.cc:1511] Processing CreateTablet for tablet 81a0d1ae8dd1431d9f241ba27f35f18f (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:14:05.251551 19004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a0d1ae8dd1431d9f241ba27f35f18f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:05.261240 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:05.270788 19003 tablet_service.cc:1511] Processing CreateTablet for tablet 20ddebbef4404377b6b25cc37866b9e2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:14:05.272778 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ddebbef4404377b6b25cc37866b9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.276501 19002 tablet_service.cc:1511] Processing CreateTablet for tablet c890b118e3b14e4291cb99b7c9f31ae2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:14:05.278330 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c890b118e3b14e4291cb99b7c9f31ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.279863 19227 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0ebf6b04c74f4c994950e78fccf2fe (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:14:05.281745 19227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0ebf6b04c74f4c994950e78fccf2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.282395 19001 tablet_service.cc:1511] Processing CreateTablet for tablet 169518fbf85c4378af0ff5af5df6dff1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:14:05.284356 19001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169518fbf85c4378af0ff5af5df6dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.283996 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 81a0d1ae8dd1431d9f241ba27f35f18f (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:14:05.284492 19225 tablet_service.cc:1511] Processing CreateTablet for tablet 169518fbf85c4378af0ff5af5df6dff1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:14:05.285801 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a0d1ae8dd1431d9f241ba27f35f18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.287665 19224 tablet_service.cc:1511] Processing CreateTablet for tablet 9c7af1ca05aa414999a70c06b637cb56 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:14:05.288452 19000 tablet_service.cc:1511] Processing CreateTablet for tablet edf86dd5411044549e507998d7b14952 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:14:05.289659 19224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c7af1ca05aa414999a70c06b637cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.290136 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf86dd5411044549e507998d7b14952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.292348 19223 tablet_service.cc:1511] Processing CreateTablet for tablet d311e304cfc84db5a4dfc30459f4d72e (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:14:05.292744 19155 tablet_service.cc:1511] Processing CreateTablet for tablet 64a4de1a07654315ae7678c1f9e8fcca (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:14:05.294055 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d311e304cfc84db5a4dfc30459f4d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.294943 19155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a4de1a07654315ae7678c1f9e8fcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.296653 19222 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff6615d74da465f9105797edb9c3698 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:14:05.305817 19226 tablet_service.cc:1511] Processing CreateTablet for tablet c890b118e3b14e4291cb99b7c9f31ae2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:14:05.298403 19221 tablet_service.cc:1511] Processing CreateTablet for tablet 0cc2ff8089f3493a8d2e92e4deb28750 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:14:05.298635 19154 tablet_service.cc:1511] Processing CreateTablet for tablet 81a0d1ae8dd1431d9f241ba27f35f18f (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:14:05.309403 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169518fbf85c4378af0ff5af5df6dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.300570 19153 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0ebf6b04c74f4c994950e78fccf2fe (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:14:05.318102 19154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a0d1ae8dd1431d9f241ba27f35f18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.319087 19153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0ebf6b04c74f4c994950e78fccf2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.319761 19221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc2ff8089f3493a8d2e92e4deb28750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.282454 19229 tablet_service.cc:1511] Processing CreateTablet for tablet fbccd6116cc5452ebe3b6f446963bcd2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:14:05.321595 19229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbccd6116cc5452ebe3b6f446963bcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.304241 18998 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff6615d74da465f9105797edb9c3698 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:14:05.323974 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff6615d74da465f9105797edb9c3698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.296746 18999 tablet_service.cc:1511] Processing CreateTablet for tablet d311e304cfc84db5a4dfc30459f4d72e (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:14:05.326285 18999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d311e304cfc84db5a4dfc30459f4d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.322530 19222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff6615d74da465f9105797edb9c3698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.328339 19226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c890b118e3b14e4291cb99b7c9f31ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.341229 19220 tablet_service.cc:1511] Processing CreateTablet for tablet d99ec37464bd4047a9485e16e72301e9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:14:05.343875 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d99ec37464bd4047a9485e16e72301e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.344977 18997 tablet_service.cc:1511] Processing CreateTablet for tablet d409ffd2084d48bf8886755a2146b5e0 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:14:05.345039 19152 tablet_service.cc:1511] Processing CreateTablet for tablet 20ddebbef4404377b6b25cc37866b9e2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:14:05.347931 19151 tablet_service.cc:1511] Processing CreateTablet for tablet 169518fbf85c4378af0ff5af5df6dff1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:14:05.348613 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ddebbef4404377b6b25cc37866b9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.349630 19151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 169518fbf85c4378af0ff5af5df6dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.354267 18997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d409ffd2084d48bf8886755a2146b5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.359968 19150 tablet_service.cc:1511] Processing CreateTablet for tablet 9c7af1ca05aa414999a70c06b637cb56 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:14:05.362685 19150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c7af1ca05aa414999a70c06b637cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.373379 19149 tablet_service.cc:1511] Processing CreateTablet for tablet edf86dd5411044549e507998d7b14952 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:14:05.373668 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 2ff6615d74da465f9105797edb9c3698 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:14:05.376264 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff6615d74da465f9105797edb9c3698. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.379556 19149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf86dd5411044549e507998d7b14952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.380652 19147 tablet_service.cc:1511] Processing CreateTablet for tablet 0cc2ff8089f3493a8d2e92e4deb28750 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:14:05.382537 19147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc2ff8089f3493a8d2e92e4deb28750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.384747 19146 tablet_service.cc:1511] Processing CreateTablet for tablet d409ffd2084d48bf8886755a2146b5e0 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:14:05.386633 19146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d409ffd2084d48bf8886755a2146b5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.386432 19081 tablet_service.cc:1511] Processing CreateTablet for tablet fbccd6116cc5452ebe3b6f446963bcd2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:14:05.387995 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbccd6116cc5452ebe3b6f446963bcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.391004 19080 tablet_service.cc:1511] Processing CreateTablet for tablet 64a4de1a07654315ae7678c1f9e8fcca (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:14:05.393060 19080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a4de1a07654315ae7678c1f9e8fcca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.396477 19079 tablet_service.cc:1511] Processing CreateTablet for tablet 3d0ebf6b04c74f4c994950e78fccf2fe (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:14:05.398393 19079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d0ebf6b04c74f4c994950e78fccf2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.400593 19078 tablet_service.cc:1511] Processing CreateTablet for tablet 20ddebbef4404377b6b25cc37866b9e2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:14:05.402488 19078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ddebbef4404377b6b25cc37866b9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.404449 19077 tablet_service.cc:1511] Processing CreateTablet for tablet c890b118e3b14e4291cb99b7c9f31ae2 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:14:05.406208 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c890b118e3b14e4291cb99b7c9f31ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.408690 19076 tablet_service.cc:1511] Processing CreateTablet for tablet 9c7af1ca05aa414999a70c06b637cb56 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:14:05.410434 19076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c7af1ca05aa414999a70c06b637cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.413249 19075 tablet_service.cc:1511] Processing CreateTablet for tablet edf86dd5411044549e507998d7b14952 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:14:05.413610 19074 tablet_service.cc:1511] Processing CreateTablet for tablet d311e304cfc84db5a4dfc30459f4d72e (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:14:05.415302 19075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf86dd5411044549e507998d7b14952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.416563 19072 tablet_service.cc:1511] Processing CreateTablet for tablet d409ffd2084d48bf8886755a2146b5e0 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:14:05.418915 19073 tablet_service.cc:1511] Processing CreateTablet for tablet 0cc2ff8089f3493a8d2e92e4deb28750 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:14:05.421028 19072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d409ffd2084d48bf8886755a2146b5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.423790 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cc2ff8089f3493a8d2e92e4deb28750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.427758 19074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d311e304cfc84db5a4dfc30459f4d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.513195 18998 tablet_service.cc:1511] Processing CreateTablet for tablet d99ec37464bd4047a9485e16e72301e9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:14:05.515184 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d99ec37464bd4047a9485e16e72301e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.536455 19041 heartbeater.cc:499] Master 127.9.11.126:34843 was elected leader, sending a full tablet report...
I20260709 12:14:05.551385 19006 tablet_service.cc:1511] Processing CreateTablet for tablet e913e5c22bd646e1944cf0bac9dae4c9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:14:05.561990 19006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e913e5c22bd646e1944cf0bac9dae4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.565482 19229 tablet_service.cc:1511] Processing CreateTablet for tablet e913e5c22bd646e1944cf0bac9dae4c9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:14:05.570026 19223 tablet_service.cc:1511] Processing CreateTablet for tablet 0741f1cc27bd423d9b9c55f4665cfce1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:14:05.583484 19149 tablet_service.cc:1511] Processing CreateTablet for tablet e913e5c22bd646e1944cf0bac9dae4c9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:14:05.584867 19229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e913e5c22bd646e1944cf0bac9dae4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.596400 19003 tablet_service.cc:1511] Processing CreateTablet for tablet 9c4c556a691543529aee3c07a4015d8d (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:14:05.598295 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c4c556a691543529aee3c07a4015d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.582316 19075 tablet_service.cc:1511] Processing CreateTablet for tablet 0741f1cc27bd423d9b9c55f4665cfce1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:14:05.562064 19073 tablet_service.cc:1511] Processing CreateTablet for tablet d99ec37464bd4047a9485e16e72301e9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:14:05.599759 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741f1cc27bd423d9b9c55f4665cfce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.592267 19081 tablet_service.cc:1511] Processing CreateTablet for tablet 9c4c556a691543529aee3c07a4015d8d (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:14:05.602133 19284 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:05.611943 19286 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:05.615486 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d99ec37464bd4047a9485e16e72301e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.622352 19149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e913e5c22bd646e1944cf0bac9dae4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.634438 19075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741f1cc27bd423d9b9c55f4665cfce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.635987 19286 tablet_bootstrap.cc:654] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.639846 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c4c556a691543529aee3c07a4015d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.645052 19004 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec43e6601ae4d8cab4e09c7e77632ae (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:14:05.651737 19004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec43e6601ae4d8cab4e09c7e77632ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.652675 19116 heartbeater.cc:499] Master 127.9.11.126:34843 was elected leader, sending a full tablet report...
I20260709 12:14:05.654578 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec43e6601ae4d8cab4e09c7e77632ae (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:14:05.657148 19152 tablet_service.cc:1511] Processing CreateTablet for tablet 0741f1cc27bd423d9b9c55f4665cfce1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:14:05.658128 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec43e6601ae4d8cab4e09c7e77632ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.665133 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 9ec43e6601ae4d8cab4e09c7e77632ae (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:14:05.658635 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0741f1cc27bd423d9b9c55f4665cfce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.640776 19284 tablet_bootstrap.cc:654] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.666858 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ec43e6601ae4d8cab4e09c7e77632ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.676025 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 9c4c556a691543529aee3c07a4015d8d (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:14:05.677187 19283 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:05.681296 19285 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:05.682703 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c4c556a691543529aee3c07a4015d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.702713 19153 tablet_service.cc:1511] Processing CreateTablet for tablet a377e1ad0954438589b509e2491c8a72 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:14:05.706112 19225 tablet_service.cc:1511] Processing CreateTablet for tablet a377e1ad0954438589b509e2491c8a72 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:14:05.707563 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a377e1ad0954438589b509e2491c8a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.707562 19153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a377e1ad0954438589b509e2491c8a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.749248 19002 tablet_service.cc:1511] Processing CreateTablet for tablet a377e1ad0954438589b509e2491c8a72 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:14:05.751152 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a377e1ad0954438589b509e2491c8a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.761502 19284 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:05.762224 19284 ts_tablet_manager.cc:1403] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.161s	user 0.011s	sys 0.019s
I20260709 12:14:05.768290 19283 tablet_bootstrap.cc:654] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.778373 19285 tablet_bootstrap.cc:654] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.769738 19284 raft_consensus.cc:359] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:05.782289 19284 raft_consensus.cc:385] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:05.782651 19284 raft_consensus.cc:740] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:05.784076 19284 consensus_queue.cc:260] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:05.778012 19147 tablet_service.cc:1511] Processing CreateTablet for tablet 489f2ef3c1464126b44ded0df97dbee1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:14:05.786556 19147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489f2ef3c1464126b44ded0df97dbee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.777868 19286 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:05.787890 19286 ts_tablet_manager.cc:1403] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.176s	user 0.020s	sys 0.000s
I20260709 12:14:05.786707 19073 tablet_service.cc:1511] Processing CreateTablet for tablet 489f2ef3c1464126b44ded0df97dbee1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:14:05.791313 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489f2ef3c1464126b44ded0df97dbee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.792249 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 489f2ef3c1464126b44ded0df97dbee1 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:14:05.793701 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489f2ef3c1464126b44ded0df97dbee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.793681 19190 heartbeater.cc:499] Master 127.9.11.126:34843 was elected leader, sending a full tablet report...
I20260709 12:14:05.805035 18998 tablet_service.cc:1511] Processing CreateTablet for tablet c3afa3fafa244643a6da11e31e841d34 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:14:05.806655 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3afa3fafa244643a6da11e31e841d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.806828 19286 raft_consensus.cc:359] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:05.807883 19286 raft_consensus.cc:385] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:05.808276 19286 raft_consensus.cc:740] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:05.809230 19286 consensus_queue.cc:260] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:05.818506 19284 ts_tablet_manager.cc:1434] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.056s	user 0.012s	sys 0.000s
I20260709 12:14:05.821036 19223 tablet_service.cc:1511] Processing CreateTablet for tablet 43888d54ff1d470a8d63fb846d1d9282 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:14:05.822427 19149 tablet_service.cc:1511] Processing CreateTablet for tablet c3afa3fafa244643a6da11e31e841d34 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:14:05.832123 19149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3afa3fafa244643a6da11e31e841d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.834444 19285 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:05.834954 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43888d54ff1d470a8d63fb846d1d9282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.835081 19285 ts_tablet_manager.cc:1403] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.154s	user 0.005s	sys 0.019s
I20260709 12:14:05.835650 19075 tablet_service.cc:1511] Processing CreateTablet for tablet c3afa3fafa244643a6da11e31e841d34 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:14:05.837175 19075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3afa3fafa244643a6da11e31e841d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.826442 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 43888d54ff1d470a8d63fb846d1d9282 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:14:05.838860 19285 raft_consensus.cc:359] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:05.839697 19285 raft_consensus.cc:385] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:05.840054 19285 raft_consensus.cc:740] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:05.841070 19285 consensus_queue.cc:260] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:05.842506 19284 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:05.839835 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43888d54ff1d470a8d63fb846d1d9282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.851377 19285 ts_tablet_manager.cc:1434] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 12:14:05.857713 19283 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:05.858332 19283 ts_tablet_manager.cc:1403] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.182s	user 0.006s	sys 0.009s
I20260709 12:14:05.861763 19283 raft_consensus.cc:359] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:05.862605 19283 raft_consensus.cc:385] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:05.864228 19283 raft_consensus.cc:740] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:05.865880 19285 tablet_bootstrap.cc:492] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:05.867820 19073 tablet_service.cc:1511] Processing CreateTablet for tablet 43888d54ff1d470a8d63fb846d1d9282 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:14:05.869385 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43888d54ff1d470a8d63fb846d1d9282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.872786 19286 ts_tablet_manager.cc:1434] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.085s	user 0.004s	sys 0.006s
I20260709 12:14:05.874760 19264 heartbeater.cc:499] Master 127.9.11.126:34843 was elected leader, sending a full tablet report...
I20260709 12:14:05.875083 19285 tablet_bootstrap.cc:654] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.878340 19286 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
W20260709 12:14:05.879348 19117 tablet.cc:2406] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:14:05.897979 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 7726c731410a492188ef1b403a54f3bc (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:14:05.900249 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7726c731410a492188ef1b403a54f3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.900782 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 7726c731410a492188ef1b403a54f3bc (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:14:05.902277 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7726c731410a492188ef1b403a54f3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.911154 19284 tablet_bootstrap.cc:654] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.915484 19283 consensus_queue.cc:260] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:05.923848 19002 tablet_service.cc:1511] Processing CreateTablet for tablet 7726c731410a492188ef1b403a54f3bc (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:14:05.924765 19147 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3e80b40cf94fca9db5cc0ea02e4c48 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:14:05.925575 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7726c731410a492188ef1b403a54f3bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.926465 19147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3e80b40cf94fca9db5cc0ea02e4c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.928687 19075 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3e80b40cf94fca9db5cc0ea02e4c48 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:14:05.930145 19075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3e80b40cf94fca9db5cc0ea02e4c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.930449 19286 tablet_bootstrap.cc:654] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:05.938077 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3e80b40cf94fca9db5cc0ea02e4c48 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:14:05.940230 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3e80b40cf94fca9db5cc0ea02e4c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.960104 19004 tablet_service.cc:1511] Processing CreateTablet for tablet c394ca853d394fdbafaa3a25b2633648 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:14:05.961740 19004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c394ca853d394fdbafaa3a25b2633648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.969378 19285 tablet_bootstrap.cc:492] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:05.970117 19285 ts_tablet_manager.cc:1403] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.104s	user 0.010s	sys 0.016s
I20260709 12:14:05.970239 19077 tablet_service.cc:1511] Processing CreateTablet for tablet c394ca853d394fdbafaa3a25b2633648 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:14:05.971827 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c394ca853d394fdbafaa3a25b2633648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:05.973285 19283 ts_tablet_manager.cc:1434] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.115s	user 0.015s	sys 0.024s
I20260709 12:14:05.974538 19283 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:05.982867 19290 raft_consensus.cc:493] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.983603 19290 raft_consensus.cc:515] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:05.964313 19295 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.984617 19295 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:05.986649 19290 leader_election.cc:290] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:06.001184 19284 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:06.002053 19284 ts_tablet_manager.cc:1403] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.160s	user 0.017s	sys 0.030s
I20260709 12:14:06.010185 19283 tablet_bootstrap.cc:654] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.012816 19286 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:06.016678 19286 ts_tablet_manager.cc:1403] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.139s	user 0.007s	sys 0.020s
I20260709 12:14:06.019794 19295 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:06.028817 19293 raft_consensus.cc:493] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.015398 19285 raft_consensus.cc:359] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.034587 19285 raft_consensus.cc:385] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.036164 19285 raft_consensus.cc:740] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.041946 19285 consensus_queue.cc:260] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.043334 19293 raft_consensus.cc:515] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:06.046957 19286 raft_consensus.cc:359] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:06.047827 19286 raft_consensus.cc:385] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.048167 19286 raft_consensus.cc:740] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.049039 19286 consensus_queue.cc:260] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:06.049808 19293 leader_election.cc:290] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:06.065019 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 428f9c2176004b6aa007c6a7af377fcb (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:14:06.066736 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428f9c2176004b6aa007c6a7af377fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.056391 19292 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.067571 19292 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.070619 19292 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:06.089943 19286 ts_tablet_manager.cc:1434] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:14:06.077251 19293 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.106675 19149 tablet_service.cc:1511] Processing CreateTablet for tablet c394ca853d394fdbafaa3a25b2633648 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:14:06.107549 19284 raft_consensus.cc:359] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.108501 19284 raft_consensus.cc:385] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.108832 19284 raft_consensus.cc:740] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.108523 19149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c394ca853d394fdbafaa3a25b2633648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.127946 19286 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:06.130585 19223 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1280ae7e994dc59cf16f8ddef18b72 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:14:06.132664 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1280ae7e994dc59cf16f8ddef18b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.133502 19283 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:06.134295 19283 ts_tablet_manager.cc:1403] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.160s	user 0.013s	sys 0.017s
I20260709 12:14:06.138533 19286 tablet_bootstrap.cc:654] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.121186 19284 consensus_queue.cc:260] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.111058 19293 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.163842 19228 tablet_service.cc:1511] Processing CreateTablet for tablet b9af570d255149c5aedd8856d3698e12 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:14:06.166043 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9af570d255149c5aedd8856d3698e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.147748 19285 ts_tablet_manager.cc:1434] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.177s	user 0.011s	sys 0.006s
I20260709 12:14:06.182163 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1280ae7e994dc59cf16f8ddef18b72 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:14:06.183933 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1280ae7e994dc59cf16f8ddef18b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.205690 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:06.209407 19129 leader_election.cc:343] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.198583 19293 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:06.214243 19285 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:06.223912 19002 tablet_service.cc:1511] Processing CreateTablet for tablet 428f9c2176004b6aa007c6a7af377fcb (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:14:06.225639 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428f9c2176004b6aa007c6a7af377fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.233027 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1280ae7e994dc59cf16f8ddef18b72 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:14:06.235033 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1280ae7e994dc59cf16f8ddef18b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.236375 19284 ts_tablet_manager.cc:1434] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.234s	user 0.010s	sys 0.000s
I20260709 12:14:06.236186 19003 tablet_service.cc:1511] Processing CreateTablet for tablet 55dd90070791474ea046bd86b13e3f75 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:14:06.237735 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55dd90070791474ea046bd86b13e3f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.249099 19225 tablet_service.cc:1511] Processing CreateTablet for tablet 5373fb94420c4369bab88209ddbd5324 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:14:06.250756 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5373fb94420c4369bab88209ddbd5324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.251397 19283 raft_consensus.cc:359] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.253737 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 428f9c2176004b6aa007c6a7af377fcb (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:14:06.268174 19283 raft_consensus.cc:385] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.280537 19075 tablet_service.cc:1511] Processing CreateTablet for tablet b9af570d255149c5aedd8856d3698e12 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:14:06.284931 19284 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:06.285681 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:06.291540 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:06.292537 19091 raft_consensus.cc:2468] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:06.294046 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:06.294440 18980 leader_election.cc:304] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:06.257541 19147 tablet_service.cc:1511] Processing CreateTablet for tablet b9af570d255149c5aedd8856d3698e12 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:14:06.295563 19290 raft_consensus.cc:2804] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:06.295998 19290 raft_consensus.cc:493] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:06.296144 19130 leader_election.cc:343] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.296358 19147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9af570d255149c5aedd8856d3698e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.296461 19290 raft_consensus.cc:3060] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.296759 19130 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:06.298305 19295 raft_consensus.cc:2749] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:06.303875 19075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9af570d255149c5aedd8856d3698e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.303682 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:06.257541 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
W20260709 12:14:06.308243 19203 leader_election.cc:343] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.275550 19283 raft_consensus.cc:740] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.309056 19073 tablet_service.cc:1511] Processing CreateTablet for tablet 55dd90070791474ea046bd86b13e3f75 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:14:06.309355 19081 tablet_service.cc:1511] Processing CreateTablet for tablet 5373fb94420c4369bab88209ddbd5324 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:14:06.309930 19283 consensus_queue.cc:260] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.311604 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:06.314343 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:06.315472 19283 ts_tablet_manager.cc:1434] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.181s	user 0.004s	sys 0.007s
W20260709 12:14:06.316179 19055 leader_election.cc:343] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.316825 19283 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
W20260709 12:14:06.317008 19055 leader_election.cc:343] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:06.321244 18979 leader_election.cc:343] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.321831 19152 tablet_service.cc:1511] Processing CreateTablet for tablet b4e5cc9ac6c94ccda322218e1f7eb2cb (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:14:06.323370 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e5cc9ac6c94ccda322218e1f7eb2cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.259090 19153 tablet_service.cc:1511] Processing CreateTablet for tablet 55dd90070791474ea046bd86b13e3f75 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:14:06.323925 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 428f9c2176004b6aa007c6a7af377fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.258875 18998 tablet_service.cc:1511] Processing CreateTablet for tablet 5373fb94420c4369bab88209ddbd5324 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:14:06.341944 19286 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:06.325644 19153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55dd90070791474ea046bd86b13e3f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.344776 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:06.353022 19056 leader_election.cc:343] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.353237 19228 tablet_service.cc:1511] Processing CreateTablet for tablet b4e5cc9ac6c94ccda322218e1f7eb2cb (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:14:06.353638 19056 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:06.354871 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e5cc9ac6c94ccda322218e1f7eb2cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.355252 19293 raft_consensus.cc:2749] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:06.356504 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5373fb94420c4369bab88209ddbd5324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.364243 19004 tablet_service.cc:1511] Processing CreateTablet for tablet b4e5cc9ac6c94ccda322218e1f7eb2cb (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:14:06.366577 19004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e5cc9ac6c94ccda322218e1f7eb2cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.367697 19283 tablet_bootstrap.cc:654] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.371488 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55dd90070791474ea046bd86b13e3f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.372164 19286 ts_tablet_manager.cc:1403] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.246s	user 0.020s	sys 0.022s
I20260709 12:14:06.379231 19286 raft_consensus.cc:359] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.380096 19286 raft_consensus.cc:385] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.380401 19286 raft_consensus.cc:740] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.381156 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5373fb94420c4369bab88209ddbd5324. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.391441 19285 tablet_bootstrap.cc:654] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.399313 19076 tablet_service.cc:1511] Processing CreateTablet for tablet 7a1299cc26f84cbaa6278121975f7961 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:14:06.401055 19076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a1299cc26f84cbaa6278121975f7961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.409457 19290 raft_consensus.cc:515] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:06.422891 19319 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.424484 19149 tablet_service.cc:1511] Processing CreateTablet for tablet 7a1299cc26f84cbaa6278121975f7961 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:14:06.427356 19319 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.428421 19149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a1299cc26f84cbaa6278121975f7961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.431473 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:06.438367 19319 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
W20260709 12:14:06.438825 18979 leader_election.cc:343] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.447028 19284 tablet_bootstrap.cc:654] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.446943 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:06.447445 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:06.448364 19016 raft_consensus.cc:2393] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 3fcac1aec8df4c599b4c67550ce347e1 in this term.
W20260709 12:14:06.449023 18979 leader_election.cc:343] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.449545 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:06.449775 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:06.450215 19090 raft_consensus.cc:3060] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.452508 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 7a1299cc26f84cbaa6278121975f7961 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:14:06.454105 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a1299cc26f84cbaa6278121975f7961. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.456672 19055 leader_election.cc:304] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:06.460775 19003 tablet_service.cc:1511] Processing CreateTablet for tablet 0eed4a045f6747428301e56cf20644b7 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:14:06.463116 19292 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.463275 19075 tablet_service.cc:1511] Processing CreateTablet for tablet 0eed4a045f6747428301e56cf20644b7 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:14:06.463721 19292 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:06.465119 19075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eed4a045f6747428301e56cf20644b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.466217 19292 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:06.467267 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:06.436363 19286 consensus_queue.cc:260] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.459990 19290 leader_election.cc:290] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:06.474357 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eed4a045f6747428301e56cf20644b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.480737 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 20a717f41e3045c8b600ca072c4b1efd (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:14:06.485317 19239 raft_consensus.cc:2468] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
W20260709 12:14:06.472043 19203 leader_election.cc:343] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.486641 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a717f41e3045c8b600ca072c4b1efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.474180 19225 tablet_service.cc:1511] Processing CreateTablet for tablet 20a717f41e3045c8b600ca072c4b1efd (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:14:06.490028 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a717f41e3045c8b600ca072c4b1efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.490312 18980 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:06.491869 19319 raft_consensus.cc:2804] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:06.491762 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:06.492431 19319 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.492818 19319 raft_consensus.cc:3060] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.496307 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
W20260709 12:14:06.498093 19204 leader_election.cc:343] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.498629 19204 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:06.500262 19283 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:06.500941 19283 ts_tablet_manager.cc:1403] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.184s	user 0.020s	sys 0.011s
I20260709 12:14:06.496352 19090 raft_consensus.cc:2468] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:06.509096 19293 raft_consensus.cc:2749] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:06.471822 19152 tablet_service.cc:1511] Processing CreateTablet for tablet 0eed4a045f6747428301e56cf20644b7 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:14:06.513644 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eed4a045f6747428301e56cf20644b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.513409 19002 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecdce0779954a328062dfca1a0364e4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:14:06.513672 19223 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecdce0779954a328062dfca1a0364e4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:14:06.514946 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecdce0779954a328062dfca1a0364e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.515486 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecdce0779954a328062dfca1a0364e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.508164 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 20a717f41e3045c8b600ca072c4b1efd (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:14:06.521467 18980 leader_election.cc:304] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:06.521343 19074 tablet_service.cc:1511] Processing CreateTablet for tablet 24dc0ead3da3436d97931d4aff80a357 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:14:06.521468 19292 raft_consensus.cc:2749] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:06.532327 19283 raft_consensus.cc:359] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.533471 19290 raft_consensus.cc:2804] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:06.535899 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a717f41e3045c8b600ca072c4b1efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.548388 19319 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.548852 19154 tablet_service.cc:1511] Processing CreateTablet for tablet 24dc0ead3da3436d97931d4aff80a357 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:14:06.550552 19154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0ead3da3436d97931d4aff80a357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.552152 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:06.554836 19074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0ead3da3436d97931d4aff80a357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.558763 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecdce0779954a328062dfca1a0364e4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:14:06.559211 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 24dc0ead3da3436d97931d4aff80a357 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:14:06.560519 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecdce0779954a328062dfca1a0364e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.560770 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24dc0ead3da3436d97931d4aff80a357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.542622 19283 raft_consensus.cc:385] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.571245 19284 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:06.571964 19284 ts_tablet_manager.cc:1403] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.302s	user 0.023s	sys 0.002s
I20260709 12:14:06.580137 18997 tablet_service.cc:1511] Processing CreateTablet for tablet e8e8cd17ef794dd18367b9052aae4c29 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:14:06.581884 18997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8e8cd17ef794dd18367b9052aae4c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.576288 19290 raft_consensus.cc:697] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:06.588248 19284 raft_consensus.cc:359] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.589214 19284 raft_consensus.cc:385] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.589565 19284 raft_consensus.cc:740] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
W20260709 12:14:06.595132 19204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 97281.2us
W20260709 12:14:06.596395 19204 leader_election.cc:343] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.596968 19204 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:06.598137 19323 raft_consensus.cc:2749] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:06.598062 19290 consensus_queue.cc:237] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:06.604802 19286 ts_tablet_manager.cc:1434] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.232s	user 0.008s	sys 0.002s
I20260709 12:14:06.605542 19285 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:06.606236 19285 ts_tablet_manager.cc:1403] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.395s	user 0.009s	sys 0.016s
I20260709 12:14:06.610304 19285 raft_consensus.cc:359] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.590435 19284 consensus_queue.cc:260] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.611240 19285 raft_consensus.cc:385] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.611694 19285 raft_consensus.cc:740] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.612891 19285 consensus_queue.cc:260] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.622785 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:06.623600 19239 raft_consensus.cc:3060] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.626735 19285 ts_tablet_manager.cc:1434] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:14:06.614239 19286 tablet_bootstrap.cc:492] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:06.633536 19153 tablet_service.cc:1511] Processing CreateTablet for tablet e8e8cd17ef794dd18367b9052aae4c29 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:14:06.636947 19153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8e8cd17ef794dd18367b9052aae4c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.637434 18913 catalog_manager.cc:5697] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:06.572677 19283 raft_consensus.cc:740] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.643167 19283 consensus_queue.cc:260] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.644409 19081 tablet_service.cc:1511] Processing CreateTablet for tablet e8e8cd17ef794dd18367b9052aae4c29 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:14:06.645604 19165 raft_consensus.cc:3060] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.645927 19283 ts_tablet_manager.cc:1434] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.145s	user 0.011s	sys 0.000s
I20260709 12:14:06.649119 19239 raft_consensus.cc:2468] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:06.650652 18980 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:06.628809 19319 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:06.651867 19319 raft_consensus.cc:2804] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:06.653626 19319 raft_consensus.cc:697] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:06.654093 19285 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:06.654628 19319 consensus_queue.cc:237] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:06.655213 19223 tablet_service.cc:1511] Processing CreateTablet for tablet 718f20f68825449c836f67bd5a4bae55 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:14:06.659325 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718f20f68825449c836f67bd5a4bae55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.662237 19295 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.664860 19325 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.664812 19295 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.665680 19325 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.667080 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8e8cd17ef794dd18367b9052aae4c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.667414 19073 tablet_service.cc:1511] Processing CreateTablet for tablet 718f20f68825449c836f67bd5a4bae55 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:14:06.668907 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718f20f68825449c836f67bd5a4bae55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.672050 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:06.672816 19090 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:06.672923 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:06.673552 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:06.660382 19283 tablet_bootstrap.cc:492] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:06.674259 19295 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:06.675935 19130 leader_election.cc:343] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.662889 18997 tablet_service.cc:1511] Processing CreateTablet for tablet 718f20f68825449c836f67bd5a4bae55 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:14:06.677548 18997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718f20f68825449c836f67bd5a4bae55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.675827 18980 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:06.675156 19325 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:06.681107 19325 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:06.681465 19325 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.681944 19325 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.674350 19239 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:06.675055 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:06.697901 19129 leader_election.cc:343] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:06.698537 19129 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:06.699983 19295 raft_consensus.cc:2749] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:06.701985 19165 raft_consensus.cc:2468] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:06.710839 19285 tablet_bootstrap.cc:654] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.719866 19003 tablet_service.cc:1511] Processing CreateTablet for tablet 9061504a72094d2abbe26a00b4998e9c (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:14:06.720917 19286 tablet_bootstrap.cc:654] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.720264 19154 tablet_service.cc:1511] Processing CreateTablet for tablet 9061504a72094d2abbe26a00b4998e9c (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:14:06.721621 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9061504a72094d2abbe26a00b4998e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.722216 19154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9061504a72094d2abbe26a00b4998e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.726415 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 9061504a72094d2abbe26a00b4998e9c (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:14:06.727942 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9061504a72094d2abbe26a00b4998e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.724650 19284 ts_tablet_manager.cc:1434] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.151s	user 0.013s	sys 0.006s
I20260709 12:14:06.733538 19284 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:06.746428 19074 tablet_service.cc:1511] Processing CreateTablet for tablet f9eff763afc14d8aa7bbd4505c26d0c3 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:14:06.747056 19284 tablet_bootstrap.cc:654] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.747969 19283 tablet_bootstrap.cc:654] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.759745 19074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9eff763afc14d8aa7bbd4505c26d0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.760826 19148 tablet_service.cc:1511] Processing CreateTablet for tablet f9eff763afc14d8aa7bbd4505c26d0c3 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:14:06.762420 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9eff763afc14d8aa7bbd4505c26d0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.778733 19228 tablet_service.cc:1511] Processing CreateTablet for tablet f9eff763afc14d8aa7bbd4505c26d0c3 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:14:06.780566 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9eff763afc14d8aa7bbd4505c26d0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.786871 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 54df033d2cbe422390e0fa4bf287ee14 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:14:06.788991 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54df033d2cbe422390e0fa4bf287ee14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.788573 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 54df033d2cbe422390e0fa4bf287ee14 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:14:06.790764 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54df033d2cbe422390e0fa4bf287ee14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.790522 19325 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.793673 19325 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:06.794693 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:06.795393 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:06.795570 19089 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.808584 19239 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:06.819231 19152 tablet_service.cc:1511] Processing CreateTablet for tablet 54df033d2cbe422390e0fa4bf287ee14 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:14:06.820951 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54df033d2cbe422390e0fa4bf287ee14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.824774 19089 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:06.837075 19286 tablet_bootstrap.cc:492] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
W20260709 12:14:06.852845 18898 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 45340.7us
I20260709 12:14:06.887827 18980 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:06.889312 19325 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:06.889919 19325 raft_consensus.cc:697] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:06.890314 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 62d133466fbe4df8a6e09156cec6acf4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:14:06.890681 19002 tablet_service.cc:1511] Processing CreateTablet for tablet 62d133466fbe4df8a6e09156cec6acf4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:14:06.890869 19325 consensus_queue.cc:237] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.891939 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d133466fbe4df8a6e09156cec6acf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.892163 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d133466fbe4df8a6e09156cec6acf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.884419 19286 ts_tablet_manager.cc:1403] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.270s	user 0.015s	sys 0.025s
I20260709 12:14:06.892211 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 62d133466fbe4df8a6e09156cec6acf4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:14:06.895980 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d133466fbe4df8a6e09156cec6acf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.897038 19285 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:06.897637 19285 ts_tablet_manager.cc:1403] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.244s	user 0.006s	sys 0.013s
I20260709 12:14:06.898245 19286 raft_consensus.cc:359] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.898773 19283 tablet_bootstrap.cc:492] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:06.903930 19286 raft_consensus.cc:385] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.904281 19283 ts_tablet_manager.cc:1403] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.257s	user 0.038s	sys 0.057s
I20260709 12:14:06.904753 19285 raft_consensus.cc:359] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.905548 19285 raft_consensus.cc:385] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.905880 19285 raft_consensus.cc:740] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.904312 19286 raft_consensus.cc:740] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.907850 19283 raft_consensus.cc:359] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.913085 19286 consensus_queue.cc:260] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.917268 19284 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:06.917934 19284 ts_tablet_manager.cc:1403] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.185s	user 0.016s	sys 0.006s
I20260709 12:14:06.912269 19283 raft_consensus.cc:385] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.919407 19283 raft_consensus.cc:740] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.920399 19283 consensus_queue.cc:260] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.919221 19285 consensus_queue.cc:260] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.906637 19239 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:06.924119 19286 ts_tablet_manager.cc:1434] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.003s
I20260709 12:14:06.933535 19286 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:06.933794 19283 ts_tablet_manager.cc:1434] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.001s
I20260709 12:14:06.935187 19283 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:06.935967 19148 tablet_service.cc:1511] Processing CreateTablet for tablet bee0ab48d58247cf8e32af4e5a43e66a (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:14:06.937496 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee0ab48d58247cf8e32af4e5a43e66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.924889 19284 raft_consensus.cc:359] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.952530 19223 tablet_service.cc:1511] Processing CreateTablet for tablet bee0ab48d58247cf8e32af4e5a43e66a (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:14:06.953846 19283 tablet_bootstrap.cc:654] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:06.954289 19223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee0ab48d58247cf8e32af4e5a43e66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:06.963757 19295 raft_consensus.cc:493] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.964383 19295 raft_consensus.cc:515] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:06.970528 19285 ts_tablet_manager.cc:1434] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.073s	user 0.011s	sys 0.001s
I20260709 12:14:06.971431 19295 leader_election.cc:290] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:06.971903 19285 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:06.950168 19284 raft_consensus.cc:385] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:06.975786 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0ebf6b04c74f4c994950e78fccf2fe" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:06.979250 19284 raft_consensus.cc:740] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:06.981631 19284 consensus_queue.cc:260] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:06.989178 19284 ts_tablet_manager.cc:1434] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.071s	user 0.003s	sys 0.006s
I20260709 12:14:06.992874 19284 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:06.991658 19286 tablet_bootstrap.cc:654] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.014323 19073 tablet_service.cc:1511] Processing CreateTablet for tablet eff579fc38a94049a77efcb07b6bf35c (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:14:07.016486 19073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff579fc38a94049a77efcb07b6bf35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.016402 19325 consensus_queue.cc:1048] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:07.021641 19220 tablet_service.cc:1511] Processing CreateTablet for tablet eff579fc38a94049a77efcb07b6bf35c (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:14:07.024111 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff579fc38a94049a77efcb07b6bf35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.027128 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 2b5f4455e6964b99b522e009ca88b6d8 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:14:07.028926 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5f4455e6964b99b522e009ca88b6d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.007366 19285 tablet_bootstrap.cc:654] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.037878 19284 tablet_bootstrap.cc:654] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.050724 19003 tablet_service.cc:1511] Processing CreateTablet for tablet bee0ab48d58247cf8e32af4e5a43e66a (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:14:07.051775 19154 tablet_service.cc:1511] Processing CreateTablet for tablet eff579fc38a94049a77efcb07b6bf35c (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:14:07.052577 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bee0ab48d58247cf8e32af4e5a43e66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.053264 19154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff579fc38a94049a77efcb07b6bf35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.056308 18997 tablet_service.cc:1511] Processing CreateTablet for tablet 2b5f4455e6964b99b522e009ca88b6d8 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:14:07.057890 18997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5f4455e6964b99b522e009ca88b6d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.060493 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 2b5f4455e6964b99b522e009ca88b6d8 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:14:07.059947 18911 catalog_manager.cc:5697] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:07.061934 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5f4455e6964b99b522e009ca88b6d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.062999 18911 catalog_manager.cc:5697] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:07.082019 19285 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:07.082813 19285 ts_tablet_manager.cc:1403] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.111s	user 0.003s	sys 0.015s
I20260709 12:14:07.086805 19285 raft_consensus.cc:359] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.087684 19285 raft_consensus.cc:385] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.088035 19285 raft_consensus.cc:740] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.089244 19285 consensus_queue.cc:260] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:07.093149 19130 leader_election.cc:343] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.097705 19283 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:07.098385 19283 ts_tablet_manager.cc:1403] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.163s	user 0.018s	sys 0.038s
I20260709 12:14:07.101794 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0ebf6b04c74f4c994950e78fccf2fe" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:07.102641 19239 raft_consensus.cc:2468] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:07.107779 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 9de7d5ebe62f4f1198a4076ef2c81d34 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:14:07.107990 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 9de7d5ebe62f4f1198a4076ef2c81d34 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:14:07.108920 19074 tablet_service.cc:1511] Processing CreateTablet for tablet 9de7d5ebe62f4f1198a4076ef2c81d34 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:14:07.109475 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de7d5ebe62f4f1198a4076ef2c81d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.106356 19130 leader_election.cc:304] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:07.110675 19295 raft_consensus.cc:2804] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:07.111128 19295 raft_consensus.cc:493] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.111482 19295 raft_consensus.cc:3060] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.112035 19074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de7d5ebe62f4f1198a4076ef2c81d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.109488 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de7d5ebe62f4f1198a4076ef2c81d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.120296 19295 raft_consensus.cc:515] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:07.123864 19283 raft_consensus.cc:359] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.124733 19283 raft_consensus.cc:385] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.124814 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0ebf6b04c74f4c994950e78fccf2fe" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:07.125108 19283 raft_consensus.cc:740] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.125998 19283 consensus_queue.cc:260] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.126828 19225 tablet_service.cc:1511] Processing CreateTablet for tablet 5a3cd9e4dbb041ddb6775aa9ff22575b (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:14:07.127612 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0ebf6b04c74f4c994950e78fccf2fe" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:07.128309 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3cd9e4dbb041ddb6775aa9ff22575b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:07.128749 19130 leader_election.cc:343] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.128304 19239 raft_consensus.cc:3060] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.132597 19285 ts_tablet_manager.cc:1434] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.002s
I20260709 12:14:07.133168 19152 tablet_service.cc:1511] Processing CreateTablet for tablet 5a3cd9e4dbb041ddb6775aa9ff22575b (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:14:07.133860 19285 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:07.134626 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3cd9e4dbb041ddb6775aa9ff22575b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.135672 18998 tablet_service.cc:1511] Processing CreateTablet for tablet 5a3cd9e4dbb041ddb6775aa9ff22575b (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:14:07.137122 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3cd9e4dbb041ddb6775aa9ff22575b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.138396 19081 tablet_service.cc:1511] Processing CreateTablet for tablet 0ccbbf4c1622458faa667dc368b16eff (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:14:07.143649 19325 consensus_queue.cc:1048] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:07.144208 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ccbbf4c1622458faa667dc368b16eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.149550 19239 raft_consensus.cc:2468] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:07.122750 19295 leader_election.cc:290] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:07.167261 19130 leader_election.cc:304] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:07.177682 19153 tablet_service.cc:1511] Processing CreateTablet for tablet 0ccbbf4c1622458faa667dc368b16eff (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:14:07.178866 19224 tablet_service.cc:1511] Processing CreateTablet for tablet 0ccbbf4c1622458faa667dc368b16eff (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:14:07.179534 19153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ccbbf4c1622458faa667dc368b16eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.180377 19224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ccbbf4c1622458faa667dc368b16eff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.198307 19344 raft_consensus.cc:2804] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:07.199095 19344 raft_consensus.cc:697] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:07.199754 19164 raft_consensus.cc:3060] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.200371 19077 tablet_service.cc:1511] Processing CreateTablet for tablet caf9e5cd30214f99a71d1f25f34dae84 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:14:07.201771 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf9e5cd30214f99a71d1f25f34dae84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.200100 19344 consensus_queue.cc:237] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:07.206001 19002 tablet_service.cc:1511] Processing CreateTablet for tablet caf9e5cd30214f99a71d1f25f34dae84 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:14:07.232568 19293 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.233484 19293 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.239286 19319 consensus_queue.cc:1048] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:07.240240 19283 ts_tablet_manager.cc:1434] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.142s	user 0.026s	sys 0.001s
I20260709 12:14:07.241617 19283 tablet_bootstrap.cc:492] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:07.242935 19293 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:07.264149 19148 tablet_service.cc:1511] Processing CreateTablet for tablet caf9e5cd30214f99a71d1f25f34dae84 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:14:07.268989 19284 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:07.268870 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:07.269659 19284 ts_tablet_manager.cc:1403] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.277s	user 0.019s	sys 0.010s
I20260709 12:14:07.262115 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:07.272009 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf9e5cd30214f99a71d1f25f34dae84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:07.275876 19056 leader_election.cc:343] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.280498 19286 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:07.281181 19286 ts_tablet_manager.cc:1403] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.348s	user 0.005s	sys 0.039s
I20260709 12:14:07.281738 19284 raft_consensus.cc:359] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:07.282869 19284 raft_consensus.cc:385] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.283303 19284 raft_consensus.cc:740] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.284366 19284 consensus_queue.cc:260] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:07.287992 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf9e5cd30214f99a71d1f25f34dae84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:07.289430 19055 leader_election.cc:343] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.289958 19055 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:07.290706 19284 ts_tablet_manager.cc:1434] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:14:07.292284 19319 consensus_queue.cc:1048] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:07.292801 19293 raft_consensus.cc:2749] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:07.298703 19284 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:07.303324 19286 raft_consensus.cc:359] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.304433 19286 raft_consensus.cc:385] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.304839 19286 raft_consensus.cc:740] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.305768 19286 consensus_queue.cc:260] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:07.320008 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:07.321486 19286 ts_tablet_manager.cc:1434] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.004s
I20260709 12:14:07.322717 19286 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:07.266083 19285 tablet_bootstrap.cc:654] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.324282 19224 tablet_service.cc:1511] Processing CreateTablet for tablet c0fdf1bfb7c54f7da0aa45218c041b08 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:14:07.333667 19224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fdf1bfb7c54f7da0aa45218c041b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.335748 19283 tablet_bootstrap.cc:654] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.351300 19319 consensus_queue.cc:1048] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:07.337980 18998 tablet_service.cc:1511] Processing CreateTablet for tablet c0fdf1bfb7c54f7da0aa45218c041b08 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:14:07.356832 19077 tablet_service.cc:1511] Processing CreateTablet for tablet c0fdf1bfb7c54f7da0aa45218c041b08 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:14:07.358052 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fdf1bfb7c54f7da0aa45218c041b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.366671 19319 raft_consensus.cc:493] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.367331 19319 raft_consensus.cc:515] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:07.371937 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:07.375883 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fdf1bfb7c54f7da0aa45218c041b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.377450 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:07.377811 18979 leader_election.cc:343] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.391335 19284 tablet_bootstrap.cc:654] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:07.391808 18980 leader_election.cc:343] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.392380 18980 leader_election.cc:304] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:07.374933 19319 leader_election.cc:290] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:07.399170 19293 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.399770 19293 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.402074 19293 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:07.406839 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 6079f38db31043d2bc846667f08d7d1b (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:14:07.409039 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6079f38db31043d2bc846667f08d7d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.402793 19225 tablet_service.cc:1511] Processing CreateTablet for tablet 6079f38db31043d2bc846667f08d7d1b (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:14:07.428479 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6079f38db31043d2bc846667f08d7d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.433145 19319 raft_consensus.cc:2749] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:07.434615 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 6079f38db31043d2bc846667f08d7d1b (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:14:07.436434 19286 tablet_bootstrap.cc:654] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.438731 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:07.445358 19163 raft_consensus.cc:2468] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:07.447273 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6079f38db31043d2bc846667f08d7d1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.446928 19074 tablet_service.cc:1511] Processing CreateTablet for tablet 67045c0c43a64d4a896e1928eda8289f (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:14:07.448516 19074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67045c0c43a64d4a896e1928eda8289f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.450954 19283 tablet_bootstrap.cc:492] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:07.451171 19055 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:07.451608 19283 ts_tablet_manager.cc:1403] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.210s	user 0.025s	sys 0.023s
I20260709 12:14:07.452365 19293 raft_consensus.cc:2804] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:07.452802 19293 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.453171 19293 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.452828 19356 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.454437 19356 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.457669 19356 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:07.457926 18913 catalog_manager.cc:5697] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:07.463462 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:07.464355 19163 raft_consensus.cc:2468] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:07.481977 18979 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:07.489112 19284 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:07.489828 19284 ts_tablet_manager.cc:1403] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.191s	user 0.027s	sys 0.004s
I20260709 12:14:07.492797 19285 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:07.493427 19285 ts_tablet_manager.cc:1403] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.360s	user 0.022s	sys 0.026s
I20260709 12:14:07.493461 19284 raft_consensus.cc:359] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:07.494429 19284 raft_consensus.cc:385] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.494758 19284 raft_consensus.cc:740] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.495808 19284 consensus_queue.cc:260] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:07.497153 19357 raft_consensus.cc:2804] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:07.499931 19357 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.500290 19357 raft_consensus.cc:3060] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.501282 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 67045c0c43a64d4a896e1928eda8289f (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:14:07.502667 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67045c0c43a64d4a896e1928eda8289f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.497108 19285 raft_consensus.cc:359] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.504321 19285 raft_consensus.cc:385] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.504662 19285 raft_consensus.cc:740] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.505548 19285 consensus_queue.cc:260] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.506044 19323 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.506649 19323 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.498656 19284 ts_tablet_manager.cc:1434] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:14:07.499351 19283 raft_consensus.cc:359] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:07.509738 19283 raft_consensus.cc:385] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.510069 19283 raft_consensus.cc:740] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.510170 19284 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:07.510685 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:07.511495 19089 raft_consensus.cc:2468] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:07.512063 19357 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.514190 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:07.514544 19357 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:07.516505 19203 leader_election.cc:343] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.518064 19204 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:07.518528 19284 tablet_bootstrap.cc:654] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.511023 19283 consensus_queue.cc:260] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:07.472515 18980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 80754us
I20260709 12:14:07.521258 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 67045c0c43a64d4a896e1928eda8289f (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:14:07.522913 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67045c0c43a64d4a896e1928eda8289f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.530261 19283 ts_tablet_manager.cc:1434] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.001s
I20260709 12:14:07.511055 19293 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.533479 18998 tablet_service.cc:1511] Processing CreateTablet for tablet 04add48cb044473fbc449788540f24c9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:14:07.535907 18998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04add48cb044473fbc449788540f24c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.512562 19323 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:07.538635 19323 raft_consensus.cc:2804] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:07.538844 19286 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:07.539072 19323 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.539422 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:07.540082 19163 raft_consensus.cc:3060] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.545687 19293 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:07.547184 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:07.547906 19164 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.549593 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:07.551854 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:07.554008 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:07.555416 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:07.539470 19323 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.557787 19285 ts_tablet_manager.cc:1434] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.064s	user 0.011s	sys 0.004s
I20260709 12:14:07.539444 19286 ts_tablet_manager.cc:1403] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.217s	user 0.013s	sys 0.010s
I20260709 12:14:07.545473 19283 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:07.562659 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 04add48cb044473fbc449788540f24c9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:14:07.565268 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04add48cb044473fbc449788540f24c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.564844 19153 tablet_service.cc:1511] Processing CreateTablet for tablet 04add48cb044473fbc449788540f24c9 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:14:07.567174 19153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04add48cb044473fbc449788540f24c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:07.586807 19055 leader_election.cc:343] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.562520 19286 raft_consensus.cc:359] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:07.603535 19130 consensus_peers.cc:597] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:07.593865 19286 raft_consensus.cc:385] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:14:07.595685 19055 leader_election.cc:343] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.613070 19286 raft_consensus.cc:740] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
W20260709 12:14:07.616161 18980 leader_election.cc:343] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.618966 19000 tablet_service.cc:1511] Processing CreateTablet for tablet ee5a381266c643c9a8b1ea339f87b328 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:14:07.594705 19163 raft_consensus.cc:2468] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:07.600050 19285 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:07.636278 19344 raft_consensus.cc:493] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.635548 19225 tablet_service.cc:1511] Processing CreateTablet for tablet ee5a381266c643c9a8b1ea339f87b328 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:14:07.637013 19344 raft_consensus.cc:515] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:07.638013 18979 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:07.638535 19225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5a381266c643c9a8b1ea339f87b328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.639153 19357 raft_consensus.cc:2804] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:07.639679 19357 raft_consensus.cc:697] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:07.636615 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5a381266c643c9a8b1ea339f87b328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.640664 19357 consensus_queue.cc:237] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.620097 19284 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:07.646315 19284 ts_tablet_manager.cc:1403] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.136s	user 0.011s	sys 0.006s
W20260709 12:14:07.650317 19204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 132360us
W20260709 12:14:07.650774 18980 leader_election.cc:343] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.650471 19286 consensus_queue.cc:260] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.659221 19284 raft_consensus.cc:359] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.660131 19284 raft_consensus.cc:385] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.660496 19284 raft_consensus.cc:740] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:07.661094 19343 consensus_queue.cc:1048] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:07.662590 19344 leader_election.cc:290] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:07.665445 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7af1ca05aa414999a70c06b637cb56" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:07.667101 19283 tablet_bootstrap.cc:654] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.667522 19081 tablet_service.cc:1511] Processing CreateTablet for tablet ee5a381266c643c9a8b1ea339f87b328 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:14:07.667552 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7af1ca05aa414999a70c06b637cb56" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:07.676805 19130 leader_election.cc:343] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.706051 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 3cfc76678bd1474fb6f9826360c9e870 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:14:07.707755 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cfc76678bd1474fb6f9826360c9e870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.701050 19081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee5a381266c643c9a8b1ea339f87b328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.713481 19002 tablet_service.cc:1511] Processing CreateTablet for tablet 3cfc76678bd1474fb6f9826360c9e870 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:14:07.715184 19002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cfc76678bd1474fb6f9826360c9e870. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:07.717417 19130 leader_election.cc:343] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.712255 19285 tablet_bootstrap.cc:654] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.695130 19325 consensus_queue.cc:1048] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:14:07.732525 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 3cfc76678bd1474fb6f9826360c9e870 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:14:07.734558 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cfc76678bd1474fb6f9826360c9e870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.682509 19284 consensus_queue.cc:260] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.739236 19130 leader_election.cc:304] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:07.745221 19164 raft_consensus.cc:2468] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:07.762653 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 8747727903d14849a4d18d8676bbf505 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:14:07.764256 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8747727903d14849a4d18d8676bbf505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.759629 19344 raft_consensus.cc:2749] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:07.767707 19055 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:07.770262 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 8747727903d14849a4d18d8676bbf505 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:14:07.768872 18997 tablet_service.cc:1511] Processing CreateTablet for tablet df92e505d2454dea8eabdad38c8f36e4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:14:07.773352 19360 raft_consensus.cc:493] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.774403 18997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df92e505d2454dea8eabdad38c8f36e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.774029 19360 raft_consensus.cc:515] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.776770 19360 leader_election.cc:290] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:07.772152 19293 raft_consensus.cc:2804] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:07.778574 19293 raft_consensus.cc:697] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:07.778844 19224 tablet_service.cc:1511] Processing CreateTablet for tablet 8747727903d14849a4d18d8676bbf505 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:14:07.779728 19293 consensus_queue.cc:237] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.785919 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d409ffd2084d48bf8886755a2146b5e0" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:07.786947 19074 tablet_service.cc:1511] Processing CreateTablet for tablet df92e505d2454dea8eabdad38c8f36e4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:14:07.788554 19074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df92e505d2454dea8eabdad38c8f36e4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:14:07.790900 19055 leader_election.cc:343] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.791406 18913 catalog_manager.cc:5697] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:07.799786 19284 ts_tablet_manager.cc:1434] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.153s	user 0.005s	sys 0.004s
I20260709 12:14:07.801563 19284 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:07.766057 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 48697.3us
I20260709 12:14:07.802943 19152 tablet_service.cc:1511] Processing CreateTablet for tablet df92e505d2454dea8eabdad38c8f36e4 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:14:07.804574 19152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df92e505d2454dea8eabdad38c8f36e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.804184 19323 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:07.804567 19224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8747727903d14849a4d18d8676bbf505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.810111 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:07.812270 19203 leader_election.cc:343] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.815419 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8747727903d14849a4d18d8676bbf505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.818024 19228 tablet_service.cc:1511] Processing CreateTablet for tablet 5d792b44d6c54e1490e101d60dbebb82 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:14:07.819546 19228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d792b44d6c54e1490e101d60dbebb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.824072 19003 tablet_service.cc:1511] Processing CreateTablet for tablet 5d792b44d6c54e1490e101d60dbebb82 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:14:07.825651 19003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d792b44d6c54e1490e101d60dbebb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.839527 19286 ts_tablet_manager.cc:1434] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.280s	user 0.015s	sys 0.011s
I20260709 12:14:07.881217 19323 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:07.902561 19148 tablet_service.cc:1511] Processing CreateTablet for tablet 43f821d9236644bcb6676552b6ee4019 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:14:07.903913 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:07.904206 19148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f821d9236644bcb6676552b6ee4019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.904685 19090 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:07.908260 19220 tablet_service.cc:1511] Processing CreateTablet for tablet 43f821d9236644bcb6676552b6ee4019 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:14:07.909638 19220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f821d9236644bcb6676552b6ee4019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.911617 19286 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:07.916607 18912 catalog_manager.cc:5697] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:07.919010 19077 tablet_service.cc:1511] Processing CreateTablet for tablet 5d792b44d6c54e1490e101d60dbebb82 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:14:07.920528 19077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d792b44d6c54e1490e101d60dbebb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.924618 19283 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:07.925346 19283 ts_tablet_manager.cc:1403] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.380s	user 0.012s	sys 0.007s
I20260709 12:14:07.925756 19284 tablet_bootstrap.cc:654] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.930763 19285 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:07.931423 19285 ts_tablet_manager.cc:1403] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.341s	user 0.011s	sys 0.023s
I20260709 12:14:07.933961 19000 tablet_service.cc:1511] Processing CreateTablet for tablet 43f821d9236644bcb6676552b6ee4019 (DEFAULT_TABLE table=test-workload [id=c4eb669f972f464fa35e9a2a856fba9b]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:14:07.935381 19000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f821d9236644bcb6676552b6ee4019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:07.936375 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d409ffd2084d48bf8886755a2146b5e0" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:07.938004 19055 leader_election.cc:343] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.938563 19055 leader_election.cc:304] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:07.940451 19360 raft_consensus.cc:2749] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:07.954319 19371 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.957883 19090 raft_consensus.cc:2468] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:07.955018 19371 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:07.959475 19204 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:07.960381 19371 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:07.962163 19323 raft_consensus.cc:2804] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:07.962693 19323 raft_consensus.cc:697] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:07.962934 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:07.963356 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:07.963639 19323 consensus_queue.cc:237] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:07.965997 18980 leader_election.cc:343] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:07.968338 18979 leader_election.cc:343] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:07.968851 18979 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:07.969765 19371 raft_consensus.cc:2749] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:07.957119 19286 tablet_bootstrap.cc:654] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:07.953471 19283 raft_consensus.cc:359] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:07.998811 19283 raft_consensus.cc:385] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:07.999224 19283 raft_consensus.cc:740] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.000095 19283 consensus_queue.cc:260] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.002832 19283 ts_tablet_manager.cc:1434] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.077s	user 0.002s	sys 0.006s
I20260709 12:14:08.013887 19285 raft_consensus.cc:359] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.017789 19284 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:08.018481 19284 ts_tablet_manager.cc:1403] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.217s	user 0.012s	sys 0.006s
I20260709 12:14:08.014916 19285 raft_consensus.cc:385] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.026443 19285 raft_consensus.cc:740] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.022877 18909 catalog_manager.cc:5697] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:08.030135 19284 raft_consensus.cc:359] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.030959 19284 raft_consensus.cc:385] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.031275 19284 raft_consensus.cc:740] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.024456 19283 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.035015 19285 consensus_queue.cc:260] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.038105 19285 ts_tablet_manager.cc:1434] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.106s	user 0.009s	sys 0.000s
I20260709 12:14:08.041075 19284 consensus_queue.cc:260] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.044852 19285 tablet_bootstrap.cc:492] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:08.049664 19284 ts_tablet_manager.cc:1434] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 12:14:08.050585 19286 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:08.051322 19286 ts_tablet_manager.cc:1403] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.163s	user 0.016s	sys 0.026s
I20260709 12:14:08.051621 19284 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:08.052908 19283 tablet_bootstrap.cc:654] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.054960 19286 raft_consensus.cc:359] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.055905 19286 raft_consensus.cc:385] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.056269 19286 raft_consensus.cc:740] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.057183 19286 consensus_queue.cc:260] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.060401 19286 ts_tablet_manager.cc:1434] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:14:08.061931 19286 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:08.065188 19285 tablet_bootstrap.cc:654] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.069341 19284 tablet_bootstrap.cc:654] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.074801 19283 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:08.075709 19283 ts_tablet_manager.cc:1403] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.001s
I20260709 12:14:08.079937 19283 raft_consensus.cc:359] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.080780 19283 raft_consensus.cc:385] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.081161 19283 raft_consensus.cc:740] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.082465 19283 consensus_queue.cc:260] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:08.083601 18980 consensus_peers.cc:597] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:08.085538 19283 ts_tablet_manager.cc:1434] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:14:08.087472 19283 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.087513 19286 tablet_bootstrap.cc:654] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.094664 19284 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:08.095285 19284 ts_tablet_manager.cc:1403] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.004s
I20260709 12:14:08.095647 19283 tablet_bootstrap.cc:654] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.098632 19284 raft_consensus.cc:359] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.099485 19284 raft_consensus.cc:385] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.099781 19284 raft_consensus.cc:740] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.100587 19284 consensus_queue.cc:260] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.103650 19285 tablet_bootstrap.cc:492] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:08.104818 19285 ts_tablet_manager.cc:1403] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.013s
I20260709 12:14:08.109122 19323 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.109794 19323 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.109226 19285 raft_consensus.cc:359] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.112159 19285 raft_consensus.cc:385] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.112851 19285 raft_consensus.cc:740] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.113240 19286 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:08.114501 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.115165 19286 ts_tablet_manager.cc:1403] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.054s	user 0.024s	sys 0.010s
W20260709 12:14:08.116413 19204 leader_election.cc:343] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.117209 19323 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.117833 19283 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:08.118484 19283 ts_tablet_manager.cc:1403] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.001s
I20260709 12:14:08.119055 19286 raft_consensus.cc:359] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.119874 19286 raft_consensus.cc:385] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.120198 19286 raft_consensus.cc:740] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.121181 19286 consensus_queue.cc:260] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.122347 19283 raft_consensus.cc:359] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.123188 19283 raft_consensus.cc:385] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.114482 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.122723 19284 ts_tablet_manager.cc:1434] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:14:08.124836 19284 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:08.125221 19203 leader_election.cc:343] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.125770 19203 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:08.123648 19283 raft_consensus.cc:740] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.127020 19283 consensus_queue.cc:260] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.127497 19368 raft_consensus.cc:2749] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.127972 19323 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.128507 19323 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.132328 19323 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.134593 19323 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.131402 19283 ts_tablet_manager.cc:1434] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 12:14:08.135144 19323 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.136281 19283 tablet_bootstrap.cc:492] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.138610 19286 ts_tablet_manager.cc:1434] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:14:08.139889 19286 tablet_bootstrap.cc:492] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:08.141076 19284 tablet_bootstrap.cc:654] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.114455 19285 consensus_queue.cc:260] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.148713 19285 ts_tablet_manager.cc:1434] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.002s
I20260709 12:14:08.150053 19323 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.152917 19285 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:08.154227 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.155215 19089 raft_consensus.cc:2468] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:08.156976 19204 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:08.158672 19283 tablet_bootstrap.cc:654] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.163736 19325 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.164598 19325 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.176385 19284 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:08.177899 19284 ts_tablet_manager.cc:1403] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.003s
I20260709 12:14:08.181658 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:08.171805 19323 raft_consensus.cc:2804] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.183653 19323 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.184036 19371 consensus_queue.cc:1048] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:08.184438 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.184136 19323 raft_consensus.cc:3060] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.185199 19089 raft_consensus.cc:2468] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:08.186865 18980 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:08.187857 19371 raft_consensus.cc:2804] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.188294 19371 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.188642 19371 raft_consensus.cc:3060] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.190543 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.191358 19325 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.191380 19015 raft_consensus.cc:2468] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:08.199975 19371 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.200587 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.200834 19203 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:08.201964 19368 raft_consensus.cc:2804] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.202414 19368 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.202773 19368 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.204013 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:08.207212 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.208137 19015 raft_consensus.cc:2393] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 3fcac1aec8df4c599b4c67550ce347e1 in this term.
I20260709 12:14:08.208866 19285 tablet_bootstrap.cc:654] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.209801 19343 raft_consensus.cc:493] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.210378 19343 raft_consensus.cc:515] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.246129 19344 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.265127 19344 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.265713 19368 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.255229 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d409ffd2084d48bf8886755a2146b5e0" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.256028 19371 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.249557 19323 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:08.260246 19203 leader_election.cc:343] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.276103 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d409ffd2084d48bf8886755a2146b5e0" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.277220 19089 raft_consensus.cc:2468] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:08.277969 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:08.279032 19238 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:08.279456 19130 leader_election.cc:304] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:08.280874 19377 raft_consensus.cc:2804] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.281423 19377 raft_consensus.cc:493] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.281854 19377 raft_consensus.cc:3060] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.282238 19323 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.268417 19343 leader_election.cc:290] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.294890 19286 tablet_bootstrap.cc:654] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.296759 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:08.298105 19016 raft_consensus.cc:2393] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 3fcac1aec8df4c599b4c67550ce347e1 in this term.
I20260709 12:14:08.300484 19344 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.301882 19323 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.303169 19130 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:08.303848 19239 raft_consensus.cc:2393] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:08.304543 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:08.305289 19089 raft_consensus.cc:3060] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.306249 19344 raft_consensus.cc:2804] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.306725 19344 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.307090 19344 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.302657 19323 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.311211 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.312290 19016 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:08.342315 19285 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:08.343134 19285 ts_tablet_manager.cc:1403] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.190s	user 0.034s	sys 0.043s
I20260709 12:14:08.343366 19283 tablet_bootstrap.cc:492] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:08.343989 19283 ts_tablet_manager.cc:1403] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.208s	user 0.040s	sys 0.045s
I20260709 12:14:08.347429 19285 raft_consensus.cc:359] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.348315 19285 raft_consensus.cc:385] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.348642 19285 raft_consensus.cc:740] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.349537 19285 consensus_queue.cc:260] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
W20260709 12:14:08.350360 19129 leader_election.cc:343] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.351995 19285 ts_tablet_manager.cc:1434] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:14:08.354698 19285 tablet_bootstrap.cc:492] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
W20260709 12:14:08.343029 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:08.369498 19293 consensus_queue.cc:1048] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:14:08.374480 19283 raft_consensus.cc:359] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.375717 19283 raft_consensus.cc:385] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.375808 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.376160 19283 raft_consensus.cc:740] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.377367 19283 consensus_queue.cc:260] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.380470 19283 ts_tablet_manager.cc:1434] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:14:08.382050 19283 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.382560 19357 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.383293 19357 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.385522 19285 tablet_bootstrap.cc:654] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.385933 19357 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.397203 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:08.399070 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:08.397588 19089 raft_consensus.cc:2468] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
W20260709 12:14:08.401438 19203 leader_election.cc:343] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.403525 19204 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:08.404770 19283 tablet_bootstrap.cc:654] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.405026 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.405807 19015 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:08.406749 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.406625 19284 raft_consensus.cc:359] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.407501 19284 raft_consensus.cc:385] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.407732 19091 raft_consensus.cc:2393] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:08.407804 19284 raft_consensus.cc:740] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.408370 19203 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:08.408890 19284 consensus_queue.cc:260] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.417541 19286 tablet_bootstrap.cc:492] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:08.418063 19286 ts_tablet_manager.cc:1403] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.278s	user 0.010s	sys 0.009s
I20260709 12:14:08.411373 19323 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.428567 19284 ts_tablet_manager.cc:1434] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.249s	user 0.007s	sys 0.005s
I20260709 12:14:08.430357 19284 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:08.429339 19293 raft_consensus.cc:493] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.435688 19293 raft_consensus.cc:515] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.438380 19293 leader_election.cc:290] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:08.440042 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0ebf6b04c74f4c994950e78fccf2fe" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:08.440352 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d0ebf6b04c74f4c994950e78fccf2fe" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.442080 19055 leader_election.cc:304] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:08.443787 19293 raft_consensus.cc:2749] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.445170 19368 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.446017 19368 raft_consensus.cc:2804] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.446626 19368 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.447118 19368 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.451574 19323 raft_consensus.cc:2749] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.453881 19377 raft_consensus.cc:515] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.456704 19377 leader_election.cc:290] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:08.459012 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d409ffd2084d48bf8886755a2146b5e0" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:08.460019 19089 raft_consensus.cc:3060] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.459118 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d409ffd2084d48bf8886755a2146b5e0" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:08.463353 19015 raft_consensus.cc:3060] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.425150 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:08.468923 18980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 44655.3us
I20260709 12:14:08.471743 19392 consensus_queue.cc:1048] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:08.470777 18980 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:08.473373 19393 raft_consensus.cc:2804] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:08.474189 19393 raft_consensus.cc:697] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:08.475139 19393 consensus_queue.cc:237] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.492305 19344 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.494897 19164 raft_consensus.cc:2393] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
I20260709 12:14:08.495314 19286 raft_consensus.cc:359] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.496399 19286 raft_consensus.cc:385] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.498476 19286 raft_consensus.cc:740] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
W20260709 12:14:08.489925 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 81711.3us
I20260709 12:14:08.499725 19286 consensus_queue.cc:260] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.502417 19286 ts_tablet_manager.cc:1434] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.000s
I20260709 12:14:08.503582 19286 tablet_bootstrap.cc:492] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:08.507215 19089 raft_consensus.cc:2468] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:08.508879 19130 leader_election.cc:304] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:08.509912 19377 raft_consensus.cc:2804] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:08.510432 19377 raft_consensus.cc:697] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:08.510694 19286 tablet_bootstrap.cc:654] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.511397 19377 consensus_queue.cc:237] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.497164 19015 raft_consensus.cc:2468] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:08.522720 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:08.522943 19343 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.526068 19285 tablet_bootstrap.cc:492] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:08.528178 19285 ts_tablet_manager.cc:1403] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.174s	user 0.023s	sys 0.034s
I20260709 12:14:08.529738 19284 tablet_bootstrap.cc:654] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.544466 19015 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.527591 19016 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.547909 18911 catalog_manager.cc:5697] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:08.555411 19283 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:08.556038 19283 ts_tablet_manager.cc:1403] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.174s	user 0.015s	sys 0.047s
I20260709 12:14:08.539405 19343 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.561475 18913 catalog_manager.cc:5697] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:08.567945 19343 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:08.565871 19293 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.569193 19293 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.573909 19089 raft_consensus.cc:3060] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.582291 19285 raft_consensus.cc:359] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.584889 19293 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:08.585981 19344 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.586138 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.587047 19400 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.565156 19368 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.554201 18980 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:08.587369 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:08.590142 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:08.591480 19285 raft_consensus.cc:385] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.594432 19164 raft_consensus.cc:2468] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:08.601042 19055 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:08.601455 19014 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.605536 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:08.597962 19400 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.607582 19283 raft_consensus.cc:359] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.608394 19283 raft_consensus.cc:385] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.608688 19283 raft_consensus.cc:740] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
W20260709 12:14:08.608955 18980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 58439.7us
I20260709 12:14:08.609609 19283 consensus_queue.cc:260] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.598209 19285 raft_consensus.cc:740] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.602677 19368 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.610756 19400 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:08.595398 19343 raft_consensus.cc:493] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.616686 19285 consensus_queue.cc:260] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.614068 19239 raft_consensus.cc:2468] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:08.618132 19393 raft_consensus.cc:2749] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.620725 19016 raft_consensus.cc:2468] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:08.623711 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.620821 19343 raft_consensus.cc:515] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.635535 19283 ts_tablet_manager.cc:1434] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.079s	user 0.010s	sys 0.000s
I20260709 12:14:08.636514 19285 ts_tablet_manager.cc:1434] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.108s	user 0.005s	sys 0.005s
I20260709 12:14:08.637188 19283 tablet_bootstrap.cc:492] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.637533 19285 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:08.639382 19403 raft_consensus.cc:2804] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.640030 19091 raft_consensus.cc:2468] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:08.640664 19403 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.640802 19343 leader_election.cc:290] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.641108 19403 raft_consensus.cc:3060] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.641650 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:08.642709 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:08.649305 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.650077 19016 raft_consensus.cc:2468] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:08.651409 19129 leader_election.cc:304] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:08.652422 19343 raft_consensus.cc:2804] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.652837 19343 raft_consensus.cc:493] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.653188 19343 raft_consensus.cc:3060] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.657106 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.657940 19163 raft_consensus.cc:2468] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:08.658747 19204 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:08.659977 19398 raft_consensus.cc:2804] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.660372 19398 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.660740 19398 raft_consensus.cc:3060] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.664332 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:08.665277 19164 raft_consensus.cc:2393] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
I20260709 12:14:08.666180 19286 tablet_bootstrap.cc:492] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:08.666903 19286 ts_tablet_manager.cc:1403] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.164s	user 0.021s	sys 0.010s
I20260709 12:14:08.668123 19284 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:08.668246 19203 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:08.668689 19284 ts_tablet_manager.cc:1403] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.239s	user 0.017s	sys 0.029s
I20260709 12:14:08.669160 19392 consensus_queue.cc:1048] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:08.666183 19403 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.687232 19343 raft_consensus.cc:515] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.670472 19286 raft_consensus.cc:359] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.699103 19286 raft_consensus.cc:385] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.699774 19286 raft_consensus.cc:740] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
W20260709 12:14:08.666314 19055 consensus_peers.cc:597] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:08.700748 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:08.700798 19130 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:08.701735 19239 raft_consensus.cc:2393] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:08.703522 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:08.704010 19016 raft_consensus.cc:3060] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.707806 19285 tablet_bootstrap.cc:654] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.708467 19344 raft_consensus.cc:2749] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.695585 19014 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:08.711306 19284 raft_consensus.cc:359] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.707614 19286 consensus_queue.cc:260] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.714504 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:08.701107 19055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 62028.2us
I20260709 12:14:08.716535 19284 raft_consensus.cc:385] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.699044 19392 raft_consensus.cc:2804] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:08.708798 19343 leader_election.cc:290] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.717864 19284 raft_consensus.cc:740] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.721174 19286 ts_tablet_manager.cc:1434] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.054s	user 0.010s	sys 0.002s
I20260709 12:14:08.722921 19284 consensus_queue.cc:260] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:08.732477 19129 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:08.733633 19284 ts_tablet_manager.cc:1434] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 12:14:08.734190 19344 raft_consensus.cc:2804] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:08.734865 19284 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:08.734789 19344 raft_consensus.cc:697] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:08.736562 19344 consensus_queue.cc:237] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.739605 19286 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:08.743355 19283 tablet_bootstrap.cc:654] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.744155 19203 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:08.746527 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:08.746718 19368 raft_consensus.cc:2749] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:14:08.746546 19130 leader_election.cc:343] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.749051 19403 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:08.751346 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:08.739449 19392 raft_consensus.cc:697] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
W20260709 12:14:08.752636 19130 leader_election.cc:343] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.753140 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:08.752950 19392 consensus_queue.cc:237] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.753784 19164 raft_consensus.cc:3060] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.756819 19398 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.761039 19238 raft_consensus.cc:2393] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:08.757697 19016 raft_consensus.cc:2468] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:08.768641 19284 tablet_bootstrap.cc:654] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.769306 19129 leader_election.cc:304] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:08.764124 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:08.771426 19344 raft_consensus.cc:2804] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:08.772136 19089 raft_consensus.cc:2393] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:08.772311 19344 raft_consensus.cc:697] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:08.773064 19286 tablet_bootstrap.cc:654] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.762056 19398 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:08.773793 19344 consensus_queue.cc:237] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.776705 19164 raft_consensus.cc:2468] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:08.778158 19055 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:08.778405 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cc2ff8089f3493a8d2e92e4deb28750" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:08.779296 19403 raft_consensus.cc:2804] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:08.779501 19163 raft_consensus.cc:2393] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:08.780079 19403 raft_consensus.cc:697] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:08.781179 19403 consensus_queue.cc:237] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.781471 19203 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:08.783742 19398 raft_consensus.cc:2749] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.791076 19283 tablet_bootstrap.cc:492] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:08.791685 19283 ts_tablet_manager.cc:1403] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.155s	user 0.006s	sys 0.010s
I20260709 12:14:08.791733 19285 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:08.792363 19285 ts_tablet_manager.cc:1403] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.155s	user 0.011s	sys 0.003s
I20260709 12:14:08.798159 19411 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.799070 19411 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.807528 19283 raft_consensus.cc:359] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.808669 19283 raft_consensus.cc:385] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.807474 19285 raft_consensus.cc:359] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.809861 19285 raft_consensus.cc:385] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.812050 19285 raft_consensus.cc:740] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.813081 19285 consensus_queue.cc:260] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.813920 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.815346 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.815831 19285 ts_tablet_manager.cc:1434] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 12:14:08.816488 19411 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:08.816910 19285 tablet_bootstrap.cc:492] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:08.817761 19055 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:08.819429 19293 raft_consensus.cc:2749] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.808939 19283 raft_consensus.cc:740] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.825760 19283 consensus_queue.cc:260] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.823511 19286 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:08.827299 19286 ts_tablet_manager.cc:1403] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.096s	user 0.021s	sys 0.006s
I20260709 12:14:08.829026 19283 ts_tablet_manager.cc:1434] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 12:14:08.832036 19283 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.833452 19285 tablet_bootstrap.cc:654] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.831547 19286 raft_consensus.cc:359] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.834599 19286 raft_consensus.cc:385] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.835170 19286 raft_consensus.cc:740] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.836109 19286 consensus_queue.cc:260] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.829589 19284 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:08.837961 19284 ts_tablet_manager.cc:1403] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.007s
I20260709 12:14:08.845814 19284 raft_consensus.cc:359] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.847352 19284 raft_consensus.cc:385] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.847702 19284 raft_consensus.cc:740] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.849571 19089 raft_consensus.cc:3060] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.851836 19286 ts_tablet_manager.cc:1434] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 12:14:08.852375 19283 tablet_bootstrap.cc:654] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.853128 19286 tablet_bootstrap.cc:492] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:08.853788 18913 catalog_manager.cc:5697] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:08.860689 18910 catalog_manager.cc:5697] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:08.862653 19344 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.866945 19286 tablet_bootstrap.cc:654] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.869182 19344 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.874692 19344 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.875415 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:08.875417 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.878585 19014 raft_consensus.cc:2393] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:08.880483 19130 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:08.882561 19344 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.886921 19283 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:08.872449 18911 catalog_manager.cc:5697] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:08.887630 19283 ts_tablet_manager.cc:1403] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.022s
I20260709 12:14:08.889125 18911 catalog_manager.cc:5697] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:08.896793 19344 raft_consensus.cc:2749] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:08.848773 19284 consensus_queue.cc:260] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:08.891810 19283 raft_consensus.cc:359] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.893513 19285 tablet_bootstrap.cc:492] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:08.904861 19283 raft_consensus.cc:385] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.905200 19285 ts_tablet_manager.cc:1403] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.008s
I20260709 12:14:08.905967 19392 raft_consensus.cc:493] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.906586 19392 raft_consensus.cc:515] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.909267 19392 leader_election.cc:290] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:08.905254 19283 raft_consensus.cc:740] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.910722 19283 consensus_queue.cc:260] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.911557 19284 ts_tablet_manager.cc:1434] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.073s	user 0.009s	sys 0.000s
I20260709 12:14:08.913300 19283 ts_tablet_manager.cc:1434] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 12:14:08.913398 19284 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:08.914433 19283 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:08.916617 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:08.917469 19164 raft_consensus.cc:2468] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:08.920282 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:08.922492 19286 tablet_bootstrap.cc:492] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:08.923116 19286 ts_tablet_manager.cc:1403] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.004s
I20260709 12:14:08.923884 19283 tablet_bootstrap.cc:654] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.922530 19203 leader_election.cc:304] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
W20260709 12:14:08.929847 19203 leader_election.cc:343] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:08.930732 19398 raft_consensus.cc:2804] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:08.931397 19398 raft_consensus.cc:493] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.931814 19398 raft_consensus.cc:3060] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.908955 19285 raft_consensus.cc:359] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.933584 19285 raft_consensus.cc:385] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.934027 19285 raft_consensus.cc:740] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.935050 19285 consensus_queue.cc:260] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.941865 19285 ts_tablet_manager.cc:1434] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:14:08.943081 19285 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:08.926939 19286 raft_consensus.cc:359] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.948688 19286 raft_consensus.cc:385] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:08.949020 19286 raft_consensus.cc:740] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:08.950045 19286 consensus_queue.cc:260] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.956765 19286 ts_tablet_manager.cc:1434] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.003s
I20260709 12:14:08.957918 19286 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:08.959132 19398 raft_consensus.cc:515] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:08.962612 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:08.963543 19164 raft_consensus.cc:3060] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:08.976397 19285 tablet_bootstrap.cc:654] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:08.977223 19357 consensus_queue.cc:1048] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:08.995855 19411 raft_consensus.cc:493] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.996639 19411 raft_consensus.cc:515] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:08.998726 19393 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.999367 19393 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.000409 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7af1ca05aa414999a70c06b637cb56" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.001228 19163 raft_consensus.cc:2468] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.001705 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7af1ca05aa414999a70c06b637cb56" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.001966 19393 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:09.002476 19239 raft_consensus.cc:2468] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.003532 19055 leader_election.cc:304] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.004523 19411 leader_election.cc:290] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:09.004617 19293 raft_consensus.cc:2804] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.005184 19293 raft_consensus.cc:493] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.005570 19293 raft_consensus.cc:3060] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.009555 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.011843 19344 consensus_queue.cc:1048] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.012367 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.014199 19239 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:08.995780 19285 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:09.015282 19285 ts_tablet_manager.cc:1403] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.072s	user 0.007s	sys 0.015s
W20260709 12:14:09.015388 18980 leader_election.cc:343] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.018939 19285 raft_consensus.cc:359] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.021276 18980 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:09.022326 19293 raft_consensus.cc:515] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.024258 19393 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.024674 19393 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.025177 19393 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.027181 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7af1ca05aa414999a70c06b637cb56" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:09.027935 19163 raft_consensus.cc:3060] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.039163 19163 raft_consensus.cc:2468] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:09.041461 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c7af1ca05aa414999a70c06b637cb56" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:09.046715 19237 raft_consensus.cc:3060] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.024679 19285 raft_consensus.cc:385] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.050320 19285 raft_consensus.cc:740] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.053279 19164 raft_consensus.cc:2468] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:09.055094 19203 leader_election.cc:304] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.056348 19392 raft_consensus.cc:2804] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.057011 19392 raft_consensus.cc:697] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:09.058228 19392 consensus_queue.cc:237] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.042168 19293 leader_election.cc:290] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:09.065861 19237 raft_consensus.cc:2468] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:09.067502 19056 leader_election.cc:304] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:09.066218 19405 consensus_queue.cc:1048] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.070346 19285 consensus_queue.cc:260] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.042033 19393 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.073689 19393 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:09.079802 19411 raft_consensus.cc:2804] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.080451 19411 raft_consensus.cc:697] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:09.080319 19434 consensus_queue.cc:1048] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.080657 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:09.081554 19411 consensus_queue.cc:237] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.082108 19091 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.095626 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:09.099421 19237 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.101504 19285 ts_tablet_manager.cc:1434] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.000s
I20260709 12:14:09.102771 19285 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:09.111438 19091 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:09.128521 18980 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:09.129722 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:09.129948 19393 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.130781 19393 raft_consensus.cc:697] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:09.132121 19393 consensus_queue.cc:237] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:09.135445 19203 leader_election.cc:343] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.143172 19237 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:09.124691 19398 leader_election.cc:290] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:09.146257 19283 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.146885 19283 ts_tablet_manager.cc:1403] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.233s	user 0.082s	sys 0.110s
I20260709 12:14:09.148041 19284 tablet_bootstrap.cc:654] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.153296 19283 raft_consensus.cc:359] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.154388 19283 raft_consensus.cc:385] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.154702 19283 raft_consensus.cc:740] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.155630 19283 consensus_queue.cc:260] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.150285 19286 tablet_bootstrap.cc:654] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.158046 19283 ts_tablet_manager.cc:1434] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:14:09.159242 19283 tablet_bootstrap.cc:492] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:09.165493 19410 raft_consensus.cc:493] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.166142 19410 raft_consensus.cc:515] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.167301 19285 tablet_bootstrap.cc:654] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.168962 19410 leader_election.cc:290] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:09.169819 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741f1cc27bd423d9b9c55f4665cfce1" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.170593 19163 raft_consensus.cc:2468] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.170814 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741f1cc27bd423d9b9c55f4665cfce1" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.172152 19055 leader_election.cc:304] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.170747 19283 tablet_bootstrap.cc:654] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.173089 19411 raft_consensus.cc:2804] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.172320 19237 raft_consensus.cc:2468] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.184118 19286 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:09.184490 18912 catalog_manager.cc:5697] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:09.173537 19411 raft_consensus.cc:493] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.190431 19411 raft_consensus.cc:3060] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.189513 19284 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:09.195031 19284 ts_tablet_manager.cc:1403] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.282s	user 0.015s	sys 0.002s
W20260709 12:14:09.199255 19130 consensus_peers.cc:597] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 -> Peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Couldn't send request to peer 2c0b3983ae6c48108f6f3632116cf5a8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:09.200228 19286 ts_tablet_manager.cc:1403] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.243s	user 0.011s	sys 0.005s
I20260709 12:14:09.203948 19284 raft_consensus.cc:359] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.204952 19284 raft_consensus.cc:385] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.205214 19284 raft_consensus.cc:740] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.205595 18909 catalog_manager.cc:5697] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:09.207244 19283 tablet_bootstrap.cc:492] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.207806 19283 ts_tablet_manager.cc:1403] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.027s
I20260709 12:14:09.211073 19283 raft_consensus.cc:359] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.211738 19283 raft_consensus.cc:385] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.212704 19283 raft_consensus.cc:740] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.213094 19286 raft_consensus.cc:359] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.213459 19283 consensus_queue.cc:260] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.205926 19284 consensus_queue.cc:260] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.213218 18913 catalog_manager.cc:5697] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:09.215945 19283 ts_tablet_manager.cc:1434] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:14:09.217036 19283 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:09.213888 19286 raft_consensus.cc:385] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.219244 19286 raft_consensus.cc:740] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.220214 19286 consensus_queue.cc:260] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.222589 19286 ts_tablet_manager.cc:1434] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.003s
I20260709 12:14:09.223770 19283 tablet_bootstrap.cc:654] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.227859 19286 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:09.226107 19284 ts_tablet_manager.cc:1434] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:14:09.216094 19411 raft_consensus.cc:515] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.232875 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741f1cc27bd423d9b9c55f4665cfce1" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:09.233160 19344 consensus_queue.cc:1048] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.234489 19411 leader_election.cc:290] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:09.233533 19163 raft_consensus.cc:3060] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.235991 19284 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:09.239636 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0741f1cc27bd423d9b9c55f4665cfce1" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:09.240341 19237 raft_consensus.cc:3060] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.244203 19286 tablet_bootstrap.cc:654] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.247778 19283 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.245280 19284 tablet_bootstrap.cc:654] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.248356 19283 ts_tablet_manager.cc:1403] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 12:14:09.251837 19283 raft_consensus.cc:359] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.252614 19283 raft_consensus.cc:385] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.252925 19283 raft_consensus.cc:740] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.253690 19283 consensus_queue.cc:260] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.256500 19283 ts_tablet_manager.cc:1434] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:14:09.262468 19285 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:09.263128 19285 ts_tablet_manager.cc:1403] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.161s	user 0.009s	sys 0.008s
I20260709 12:14:09.263187 19411 consensus_queue.cc:1048] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.264945 19344 consensus_queue.cc:1048] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:09.269451 19392 consensus_queue.cc:1048] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:09.273118 19283 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:09.275205 19357 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.275902 19357 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.276428 19163 raft_consensus.cc:2468] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:09.277989 19055 leader_election.cc:304] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.278949 19411 raft_consensus.cc:2804] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.279577 19285 raft_consensus.cc:359] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.280300 19285 raft_consensus.cc:385] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.279546 19411 raft_consensus.cc:697] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:09.280612 19285 raft_consensus.cc:740] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.281417 19285 consensus_queue.cc:260] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.270733 19237 raft_consensus.cc:2468] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:09.281836 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.281786 19411 consensus_queue.cc:237] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.284776 19410 consensus_queue.cc:1048] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:09.285761 19285 ts_tablet_manager.cc:1434] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.005s
I20260709 12:14:09.286949 19285 tablet_bootstrap.cc:492] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:09.291344 19286 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:09.292133 19286 ts_tablet_manager.cc:1403] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.004s
I20260709 12:14:09.278753 19357 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.328615 19285 tablet_bootstrap.cc:654] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.330516 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.333690 18979 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:09.335204 19357 raft_consensus.cc:2749] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.337507 19286 raft_consensus.cc:359] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.319372 19410 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.327306 19405 raft_consensus.cc:493] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.341037 19286 raft_consensus.cc:385] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.344584 19286 raft_consensus.cc:740] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.348219 19286 consensus_queue.cc:260] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.355515 19393 raft_consensus.cc:493] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.356415 19393 raft_consensus.cc:515] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.364791 19286 ts_tablet_manager.cc:1434] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.002s
I20260709 12:14:09.343899 19405 raft_consensus.cc:515] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.378830 19014 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.345638 19410 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.366726 19286 tablet_bootstrap.cc:492] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:09.386924 19405 leader_election.cc:290] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:09.398396 19284 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:09.399103 19284 ts_tablet_manager.cc:1403] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.163s	user 0.039s	sys 0.044s
I20260709 12:14:09.400010 19410 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.391126 19393 leader_election.cc:290] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.408911 19284 raft_consensus.cc:359] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.409739 19284 raft_consensus.cc:385] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.410004 19284 raft_consensus.cc:740] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.411124 19284 consensus_queue.cc:260] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.402810 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.414536 19284 ts_tablet_manager.cc:1434] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 12:14:09.428413 19283 tablet_bootstrap.cc:654] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.428694 19284 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:09.414072 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:09.425275 19393 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.415403 19405 consensus_queue.cc:1048] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:14:09.411417 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:09.431653 19393 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.418148 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.433583 19164 raft_consensus.cc:2468] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
W20260709 12:14:09.434212 19129 leader_election.cc:343] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.434511 19013 raft_consensus.cc:2468] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.422941 18909 catalog_manager.cc:5697] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:09.433857 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.434268 19393 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.439692 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.440388 19163 raft_consensus.cc:2468] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
W20260709 12:14:09.440863 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 63789.4us
I20260709 12:14:09.441704 19284 tablet_bootstrap.cc:654] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.442276 19368 consensus_queue.cc:1048] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:09.444856 19055 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.445667 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.446503 19286 tablet_bootstrap.cc:654] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.447026 19454 raft_consensus.cc:2804] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.447476 19454 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.447930 19454 raft_consensus.cc:3060] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:09.448025 19130 leader_election.cc:343] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.448712 19130 leader_election.cc:304] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
W20260709 12:14:09.424165 18980 leader_election.cc:343] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.452072 19343 raft_consensus.cc:2749] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:14:09.452590 18979 leader_election.cc:343] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.453828 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.457652 18979 leader_election.cc:304] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:09.458894 19357 raft_consensus.cc:2749] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.464838 18979 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.467181 19393 raft_consensus.cc:2804] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.467710 19393 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.468135 19393 raft_consensus.cc:3060] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.469286 19405 raft_consensus.cc:493] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.469980 19405 raft_consensus.cc:515] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.472816 19405 leader_election.cc:290] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:09.474884 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489f2ef3c1464126b44ded0df97dbee1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.481288 19293 consensus_queue.cc:1048] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:14:09.482131 19130 leader_election.cc:343] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.483474 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489f2ef3c1464126b44ded0df97dbee1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.488256 19293 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.488876 19293 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.492503 19454 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.495621 19293 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:09.496698 19130 leader_election.cc:343] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.501607 19130 leader_election.cc:304] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
W20260709 12:14:09.503391 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:09.496698 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.504639 19239 raft_consensus.cc:2468] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.506110 19056 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:09.507112 19293 raft_consensus.cc:2804] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.507500 19293 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.507850 19293 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.511461 19285 tablet_bootstrap.cc:492] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:09.512112 19285 ts_tablet_manager.cc:1403] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.225s	user 0.009s	sys 0.014s
I20260709 12:14:09.511459 19393 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.515188 19405 raft_consensus.cc:2749] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.518049 19285 raft_consensus.cc:359] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.519042 19285 raft_consensus.cc:385] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.519547 19285 raft_consensus.cc:740] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.518512 19393 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.523379 19293 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.525698 19293 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:09.527534 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:09.528291 19239 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.527395 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:09.536933 19055 leader_election.cc:343] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.525805 19286 tablet_bootstrap.cc:492] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:09.537935 19286 ts_tablet_manager.cc:1403] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.172s	user 0.015s	sys 0.003s
I20260709 12:14:09.528455 19091 raft_consensus.cc:2393] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
W20260709 12:14:09.539294 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:09.521973 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:09.544188 19286 raft_consensus.cc:359] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.546001 19286 raft_consensus.cc:385] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.546407 19286 raft_consensus.cc:740] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.520325 19285 consensus_queue.cc:260] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.497069 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:09.552093 19454 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.552875 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:09.553781 19091 raft_consensus.cc:2393] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:09.553965 19286 consensus_queue.cc:260] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.558259 19283 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.558871 19283 ts_tablet_manager.cc:1403] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.286s	user 0.025s	sys 0.015s
I20260709 12:14:09.559062 19398 raft_consensus.cc:493] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.559785 19398 raft_consensus.cc:515] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.561987 19283 raft_consensus.cc:359] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.562690 19283 raft_consensus.cc:385] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.564146 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:09.567348 19398 leader_election.cc:290] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.573372 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:09.574572 19014 raft_consensus.cc:2393] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 3fcac1aec8df4c599b4c67550ce347e1 in this term.
I20260709 12:14:09.576269 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489f2ef3c1464126b44ded0df97dbee1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.577138 19091 raft_consensus.cc:2468] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:09.578508 19204 leader_election.cc:304] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
W20260709 12:14:09.579720 19055 leader_election.cc:343] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.579945 19368 raft_consensus.cc:2804] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.580420 19368 raft_consensus.cc:493] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.580785 19368 raft_consensus.cc:3060] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.581054 19055 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:09.582896 19410 raft_consensus.cc:2749] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.585639 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489f2ef3c1464126b44ded0df97dbee1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.586369 19165 raft_consensus.cc:2468] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:09.601805 19283 raft_consensus.cc:740] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.545244 19163 raft_consensus.cc:3060] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.603139 19283 consensus_queue.cc:260] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.604609 19398 consensus_queue.cc:1048] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:14:09.609786 19393 consensus_queue.cc:1048] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.610129 19293 consensus_queue.cc:1048] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:09.611867 19393 consensus_queue.cc:1048] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.621549 19286 ts_tablet_manager.cc:1434] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.001s
I20260709 12:14:09.622946 19286 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:09.625504 19285 ts_tablet_manager.cc:1434] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.000s
I20260709 12:14:09.626821 19285 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:09.621616 19239 raft_consensus.cc:2468] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:09.629827 19286 tablet_bootstrap.cc:654] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.630875 19056 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:09.632386 19293 raft_consensus.cc:2804] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.633036 19293 raft_consensus.cc:697] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:09.634143 19293 consensus_queue.cc:237] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.637918 19284 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:09.638563 19284 ts_tablet_manager.cc:1403] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.210s	user 0.041s	sys 0.047s
I20260709 12:14:09.624424 19368 raft_consensus.cc:515] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.648500 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489f2ef3c1464126b44ded0df97dbee1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:09.649173 19088 raft_consensus.cc:3060] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.650257 19284 raft_consensus.cc:359] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.651166 19284 raft_consensus.cc:385] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.651479 19284 raft_consensus.cc:740] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.652195 19284 consensus_queue.cc:260] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.654784 19284 ts_tablet_manager.cc:1434] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.008s
I20260709 12:14:09.656041 19284 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:09.644382 19283 ts_tablet_manager.cc:1434] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.085s	user 0.008s	sys 0.000s
I20260709 12:14:09.658968 19088 raft_consensus.cc:2468] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:09.669495 19204 leader_election.cc:304] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:09.670626 19398 raft_consensus.cc:2804] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.671201 19398 raft_consensus.cc:697] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:09.672175 19398 consensus_queue.cc:237] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.660025 19283 tablet_bootstrap.cc:492] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:09.662822 19284 tablet_bootstrap.cc:654] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.685854 19285 tablet_bootstrap.cc:654] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.662060 19163 raft_consensus.cc:2468] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:09.687829 18979 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:09.689429 19393 raft_consensus.cc:2804] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:09.690126 19393 raft_consensus.cc:697] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:09.691381 19393 consensus_queue.cc:237] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.681912 19368 leader_election.cc:290] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.719288 18912 catalog_manager.cc:5697] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:09.720801 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "489f2ef3c1464126b44ded0df97dbee1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:09.721637 19163 raft_consensus.cc:3060] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.715179 19368 raft_consensus.cc:493] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.734819 19368 raft_consensus.cc:515] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.742049 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.742133 19163 raft_consensus.cc:2468] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:09.749214 19286 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:09.749869 19286 ts_tablet_manager.cc:1403] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.127s	user 0.002s	sys 0.063s
I20260709 12:14:09.753098 19286 raft_consensus.cc:359] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.753811 19286 raft_consensus.cc:385] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.754055 19286 raft_consensus.cc:740] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.754738 19286 consensus_queue.cc:260] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:09.746816 19470 log.cc:927] Time spent T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:14:09.756639 19238 raft_consensus.cc:3060] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.758138 19286 ts_tablet_manager.cc:1434] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:14:09.759374 19286 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:09.762316 19293 consensus_queue.cc:1048] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:09.766595 19286 tablet_bootstrap.cc:654] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.769022 19285 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:09.777081 19285 ts_tablet_manager.cc:1403] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.151s	user 0.014s	sys 0.022s
I20260709 12:14:09.769469 18911 catalog_manager.cc:5697] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:09.771409 19284 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:09.780736 19284 ts_tablet_manager.cc:1403] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.125s	user 0.008s	sys 0.022s
I20260709 12:14:09.782327 19285 raft_consensus.cc:359] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.782747 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a0d1ae8dd1431d9f241ba27f35f18f" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:09.783258 19285 raft_consensus.cc:385] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.783622 19285 raft_consensus.cc:740] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.783933 19284 raft_consensus.cc:359] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.784734 19284 raft_consensus.cc:385] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.785235 19284 raft_consensus.cc:740] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.786347 19284 consensus_queue.cc:260] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.788791 19284 ts_tablet_manager.cc:1434] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:14:09.791167 19284 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:09.791478 19203 leader_election.cc:304] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:09.792568 19293 consensus_queue.cc:1048] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:09.775454 19368 leader_election.cc:290] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:09.774791 19283 tablet_bootstrap.cc:654] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.784571 19285 consensus_queue.cc:260] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.803664 19285 ts_tablet_manager.cc:1434] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.024s	user 0.003s	sys 0.005s
I20260709 12:14:09.808691 19285 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:09.821828 19284 tablet_bootstrap.cc:654] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.829260 19283 tablet_bootstrap.cc:492] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.829137 18910 catalog_manager.cc:5697] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:09.834875 19283 ts_tablet_manager.cc:1403] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.175s	user 0.012s	sys 0.007s
I20260709 12:14:09.847733 19285 tablet_bootstrap.cc:654] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.860540 19283 raft_consensus.cc:359] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.861177 19393 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.862172 19286 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:09.861974 19393 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.862844 19286 ts_tablet_manager.cc:1403] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.104s	user 0.027s	sys 0.003s
I20260709 12:14:09.865900 19284 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:09.866570 19284 ts_tablet_manager.cc:1403] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.076s	user 0.005s	sys 0.026s
I20260709 12:14:09.867023 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.867949 19237 raft_consensus.cc:2393] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:09.869998 19393 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:09.870378 19284 raft_consensus.cc:359] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.872190 19284 raft_consensus.cc:385] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.872666 19284 raft_consensus.cc:740] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.873008 19283 raft_consensus.cc:385] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.873651 19283 raft_consensus.cc:740] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.874039 19284 consensus_queue.cc:260] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.874845 19283 consensus_queue.cc:260] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.876860 19284 ts_tablet_manager.cc:1434] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 12:14:09.874787 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.878116 19284 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:09.878427 19283 ts_tablet_manager.cc:1434] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 12:14:09.879619 19283 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:09.886386 19283 tablet_bootstrap.cc:654] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.886188 19392 raft_consensus.cc:2749] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.887408 19286 raft_consensus.cc:359] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.888305 19286 raft_consensus.cc:385] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.888607 19286 raft_consensus.cc:740] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.889328 18980 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:09.889398 19286 consensus_queue.cc:260] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:09.891822 19393 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.892683 19286 ts_tablet_manager.cc:1434] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:14:09.893921 19286 tablet_bootstrap.cc:492] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:09.895303 19434 raft_consensus.cc:493] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.895856 19434 raft_consensus.cc:515] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.898649 19434 leader_election.cc:290] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:09.900151 19284 tablet_bootstrap.cc:654] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.901546 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.903159 19454 raft_consensus.cc:493] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.904445 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.909271 19239 raft_consensus.cc:2468] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:09.909269 19454 raft_consensus.cc:515] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.913369 19454 leader_election.cc:290] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:09.914928 18980 leader_election.cc:304] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:09.915309 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:09.916107 19163 raft_consensus.cc:2468] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:09.917080 19357 raft_consensus.cc:2804] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.917527 19357 raft_consensus.cc:493] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.914959 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:09.917954 19357 raft_consensus.cc:3060] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.918427 19014 raft_consensus.cc:2468] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
W20260709 12:14:09.919159 18980 leader_election.cc:343] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.919561 19055 leader_election.cc:304] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:09.921049 19293 raft_consensus.cc:2804] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:09.921435 19293 raft_consensus.cc:493] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.921766 19293 raft_consensus.cc:3060] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.905742 19286 tablet_bootstrap.cc:654] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.936756 19455 raft_consensus.cc:493] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.937268 19357 raft_consensus.cc:515] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.937664 19455 raft_consensus.cc:515] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.939271 19393 raft_consensus.cc:2749] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.942842 19357 leader_election.cc:290] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:09.945755 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:09.946849 19286 tablet_bootstrap.cc:492] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:09.947489 19286 ts_tablet_manager.cc:1403] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.000s
W20260709 12:14:09.947769 19130 leader_election.cc:343] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.948601 19455 leader_election.cc:290] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:09.949128 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:09.949457 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:09.949903 19238 raft_consensus.cc:3060] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:09.951040 19130 leader_election.cc:343] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:09.951494 19283 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.951614 19130 leader_election.cc:304] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:09.952769 19455 raft_consensus.cc:2749] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:09.952109 19283 ts_tablet_manager.cc:1403] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.001s
I20260709 12:14:09.950809 19286 raft_consensus.cc:359] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.956848 19286 raft_consensus.cc:385] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.957015 19285 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:09.957154 19286 raft_consensus.cc:740] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.957588 19285 ts_tablet_manager.cc:1403] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.149s	user 0.016s	sys 0.004s
I20260709 12:14:09.958063 19286 consensus_queue.cc:260] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:09.958436 19283 raft_consensus.cc:359] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.959364 19283 raft_consensus.cc:385] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.959699 19283 raft_consensus.cc:740] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.960551 19286 ts_tablet_manager.cc:1434] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:14:09.960557 19283 consensus_queue.cc:260] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.961920 19286 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:09.961444 19285 raft_consensus.cc:359] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.962261 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:09.963713 19283 ts_tablet_manager.cc:1434] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:14:09.964926 19283 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:09.962487 19285 raft_consensus.cc:385] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.968324 19285 raft_consensus.cc:740] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:09.969600 19285 consensus_queue.cc:260] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:09.972321 19283 tablet_bootstrap.cc:654] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:09.973645 19091 raft_consensus.cc:3060] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.976760 19293 raft_consensus.cc:515] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:09.978508 19284 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:09.979465 19284 ts_tablet_manager.cc:1403] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.102s	user 0.020s	sys 0.027s
I20260709 12:14:09.982432 19238 raft_consensus.cc:2468] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:09.985428 19283 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:09.997448 19285 ts_tablet_manager.cc:1434] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 12:14:09.991143 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:09.998256 19283 ts_tablet_manager.cc:1403] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.034s	user 0.023s	sys 0.008s
I20260709 12:14:09.999424 18980 leader_election.cc:304] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:09.998755 19014 raft_consensus.cc:3060] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:09.982750 19284 raft_consensus.cc:359] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.001865 19285 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:10.002123 19284 raft_consensus.cc:385] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:09.987725 19286 tablet_bootstrap.cc:654] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.002488 19016 raft_consensus.cc:3060] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.002328 19283 raft_consensus.cc:359] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.003841 19283 raft_consensus.cc:385] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.004213 19283 raft_consensus.cc:740] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.005177 19283 consensus_queue.cc:260] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.008080 19283 ts_tablet_manager.cc:1434] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:14:10.009418 19283 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:10.002440 19284 raft_consensus.cc:740] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.010285 19285 tablet_bootstrap.cc:654] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.011245 19357 raft_consensus.cc:2804] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:10.010298 19284 consensus_queue.cc:260] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.011960 19357 raft_consensus.cc:697] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:10.012951 19357 consensus_queue.cc:237] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.014604 19284 ts_tablet_manager.cc:1434] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.002s
I20260709 12:14:10.015775 19284 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:10.018662 19283 tablet_bootstrap.cc:654] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.022512 19284 tablet_bootstrap.cc:654] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.035699 19283 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:10.036454 19283 ts_tablet_manager.cc:1403] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.001s
I20260709 12:14:10.036631 19368 raft_consensus.cc:493] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.037305 19368 raft_consensus.cc:515] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.039898 19368 leader_election.cc:290] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:10.043138 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:10.043521 19283 raft_consensus.cc:359] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.044648 19283 raft_consensus.cc:385] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.045056 19283 raft_consensus.cc:740] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.046010 19283 consensus_queue.cc:260] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.046691 19285 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:10.047326 19285 ts_tablet_manager.cc:1403] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.013s
I20260709 12:14:10.048049 19283 ts_tablet_manager.cc:1434] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:14:10.046692 19286 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:10.049161 19286 ts_tablet_manager.cc:1403] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.087s	user 0.017s	sys 0.000s
I20260709 12:14:10.049582 19283 tablet_bootstrap.cc:492] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:10.050483 19285 raft_consensus.cc:359] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.051784 19285 raft_consensus.cc:385] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.052100 19285 raft_consensus.cc:740] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.052826 19285 consensus_queue.cc:260] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.054476 19285 ts_tablet_manager.cc:1434] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:14:10.055577 19285 tablet_bootstrap.cc:492] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:10.044090 19015 raft_consensus.cc:2468] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:10.059945 19283 tablet_bootstrap.cc:654] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.061118 19203 leader_election.cc:304] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:10.062227 19285 tablet_bootstrap.cc:654] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.068912 19286 raft_consensus.cc:359] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.069835 19286 raft_consensus.cc:385] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.075711 19284 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:10.076406 19284 ts_tablet_manager.cc:1403] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.061s	user 0.018s	sys 0.000s
I20260709 12:14:10.070196 19286 raft_consensus.cc:740] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.062973 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:10.080804 19286 consensus_queue.cc:260] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.081665 19163 raft_consensus.cc:2468] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:10.070966 19368 raft_consensus.cc:2804] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:10.089040 19368 raft_consensus.cc:493] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.089587 19368 raft_consensus.cc:3060] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.096568 19454 consensus_queue.cc:1048] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:10.096881 18912 catalog_manager.cc:5697] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.098537 19014 raft_consensus.cc:2468] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:10.096674 19284 raft_consensus.cc:359] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.104894 19284 raft_consensus.cc:385] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.105264 19284 raft_consensus.cc:740] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.106173 19091 raft_consensus.cc:2468] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:10.107895 19286 ts_tablet_manager.cc:1434] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 12:14:10.109084 19286 tablet_bootstrap.cc:492] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:10.110813 19284 consensus_queue.cc:260] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.115598 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:10.116071 19343 raft_consensus.cc:493] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.116302 19163 raft_consensus.cc:3060] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.116671 19343 raft_consensus.cc:515] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.104665 19293 leader_election.cc:290] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:10.121920 19055 leader_election.cc:304] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:10.123409 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:10.133685 19284 ts_tablet_manager.cc:1434] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:14:10.129622 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:10.135353 19284 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:10.136122 19129 leader_election.cc:343] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:10.137545 19130 leader_election.cc:343] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:10.138924 19130 leader_election.cc:304] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:10.140316 19455 raft_consensus.cc:2749] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:10.126273 19343 leader_election.cc:290] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:10.143615 19284 tablet_bootstrap.cc:654] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.128484 19293 raft_consensus.cc:2804] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:10.154677 19283 tablet_bootstrap.cc:492] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:10.155364 19283 ts_tablet_manager.cc:1403] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.106s	user 0.022s	sys 0.024s
I20260709 12:14:10.157258 19454 consensus_queue.cc:1048] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:10.149466 19293 raft_consensus.cc:697] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:10.159906 19293 consensus_queue.cc:237] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.161798 19286 tablet_bootstrap.cc:654] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.161981 19393 raft_consensus.cc:493] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c0b3983ae6c48108f6f3632116cf5a8)
I20260709 12:14:10.162587 19393 raft_consensus.cc:515] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.167141 19283 raft_consensus.cc:359] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.167918 19283 raft_consensus.cc:385] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.168208 19283 raft_consensus.cc:740] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.169544 19368 raft_consensus.cc:515] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.167727 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:10.172681 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:10.173038 19163 raft_consensus.cc:2468] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:10.169509 19283 consensus_queue.cc:260] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.175577 19164 raft_consensus.cc:3060] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.177805 19283 ts_tablet_manager.cc:1434] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:14:10.179080 19283 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:10.181023 19393 leader_election.cc:290] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:10.184589 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:10.185302 19013 raft_consensus.cc:3060] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.185743 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:10.185693 19285 tablet_bootstrap.cc:492] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:10.187085 19343 raft_consensus.cc:493] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.187779 19343 raft_consensus.cc:515] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.190516 19343 leader_election.cc:290] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:10.191012 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c394ca853d394fdbafaa3a25b2633648" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:10.191318 19285 ts_tablet_manager.cc:1403] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.136s	user 0.012s	sys 0.021s
I20260709 12:14:10.191706 19015 raft_consensus.cc:2468] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:10.193171 19129 leader_election.cc:304] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:10.194540 19343 raft_consensus.cc:2804] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:10.194963 19343 raft_consensus.cc:493] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.195474 19343 raft_consensus.cc:3060] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.198135 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c394ca853d394fdbafaa3a25b2633648" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:10.198949 19088 raft_consensus.cc:2468] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:10.194810 19285 raft_consensus.cc:359] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.199734 19285 raft_consensus.cc:385] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.200037 19285 raft_consensus.cc:740] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.204332 19285 consensus_queue.cc:260] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.205230 18980 leader_election.cc:304] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:10.206825 19357 consensus_queue.cc:1048] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:10.207556 19013 raft_consensus.cc:2468] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:10.209110 19203 leader_election.cc:304] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:10.209365 19285 ts_tablet_manager.cc:1434] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:14:10.209501 19357 raft_consensus.cc:2749] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:10.210168 19398 raft_consensus.cc:2804] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:10.210629 19285 tablet_bootstrap.cc:492] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:10.210754 19398 raft_consensus.cc:697] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:10.211864 19398 consensus_queue.cc:237] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.213017 19283 tablet_bootstrap.cc:654] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.215938 19368 leader_election.cc:290] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:10.250555 19392 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.251327 19392 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.253140 19343 raft_consensus.cc:515] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.253926 19392 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:10.269037 19368 raft_consensus.cc:493] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.294447 19285 tablet_bootstrap.cc:654] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:10.296451 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:10.314610 19455 raft_consensus.cc:493] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.321561 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c394ca853d394fdbafaa3a25b2633648" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:10.322543 19088 raft_consensus.cc:3060] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.304456 19343 leader_election.cc:290] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:10.290937 19368 raft_consensus.cc:515] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.336721 19286 tablet_bootstrap.cc:492] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:10.395066 19286 ts_tablet_manager.cc:1403] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.286s	user 0.018s	sys 0.010s
I20260709 12:14:10.315675 19455 raft_consensus.cc:515] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:10.339774 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c394ca853d394fdbafaa3a25b2633648" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:10.424532 19284 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:10.425423 19284 ts_tablet_manager.cc:1403] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.290s	user 0.015s	sys 0.010s
I20260709 12:14:10.330816 19164 raft_consensus.cc:2468] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:10.430109 19368 leader_election.cc:290] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:10.356472 19357 consensus_queue.cc:1048] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.058s
W20260709 12:14:10.437312 19203 leader_election.cc:343] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:10.441498 19013 raft_consensus.cc:3060] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.459653 19283 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:10.460484 19283 ts_tablet_manager.cc:1403] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.282s	user 0.016s	sys 0.016s
I20260709 12:14:10.523463 19283 raft_consensus.cc:359] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.523991 19285 tablet_bootstrap.cc:492] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:10.539780 19285 ts_tablet_manager.cc:1403] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.329s	user 0.033s	sys 0.041s
I20260709 12:14:10.528455 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:10.544770 19283 raft_consensus.cc:385] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.545992 19283 raft_consensus.cc:740] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
W20260709 12:14:10.550117 19204 leader_election.cc:343] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:10.548327 19283 consensus_queue.cc:260] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.550848 19204 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:10.548240 18913 catalog_manager.cc:5697] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.545328 19285 raft_consensus.cc:359] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.554100 19285 raft_consensus.cc:385] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.554517 19285 raft_consensus.cc:740] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.555296 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:10.555979 19285 consensus_queue.cc:260] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.562956 19283 ts_tablet_manager.cc:1434] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.102s	user 0.007s	sys 0.003s
I20260709 12:14:10.563280 19392 raft_consensus.cc:2749] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:10.545198 19504 raft_consensus.cc:493] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.572322 19283 tablet_bootstrap.cc:492] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:10.572186 19285 ts_tablet_manager.cc:1434] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.002s
I20260709 12:14:10.574091 19504 raft_consensus.cc:515] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.583738 19285 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:10.584925 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:10.585793 19238 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:10.587296 19056 leader_election.cc:304] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:10.588231 19454 raft_consensus.cc:2804] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:10.589850 19454 raft_consensus.cc:493] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.590188 19454 raft_consensus.cc:3060] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:10.594274 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:10.583681 19013 raft_consensus.cc:2468] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:10.457724 19088 raft_consensus.cc:2468] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:10.718925 19502 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.632647 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e5cc9ac6c94ccda322218e1f7eb2cb" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:10.721472 19502 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.588030 19504 leader_election.cc:290] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
W20260709 12:14:10.624776 19204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 74682.2us
I20260709 12:14:10.638994 19129 leader_election.cc:304] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:10.728996 19343 raft_consensus.cc:2804] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:10.729583 19343 raft_consensus.cc:697] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:10.730511 19343 consensus_queue.cc:237] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.630704 19455 leader_election.cc:290] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:10.634400 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:10.731379 19454 raft_consensus.cc:515] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.743932 19454 leader_election.cc:290] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:10.735723 19502 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:10.745415 19283 tablet_bootstrap.cc:654] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:10.745760 18980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 317833us
W20260709 12:14:10.745245 19129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 106423us
I20260709 12:14:10.751037 19285 tablet_bootstrap.cc:654] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:10.751735 19129 leader_election.cc:343] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:10.780326 18913 catalog_manager.cc:5697] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.709983 19284 raft_consensus.cc:359] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.792742 19284 raft_consensus.cc:385] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.793181 19284 raft_consensus.cc:740] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.709082 19286 raft_consensus.cc:359] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.788116 19089 raft_consensus.cc:2393] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:10.798947 19286 raft_consensus.cc:385] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.781858 19357 consensus_queue.cc:1048] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:14:10.801275 19284 consensus_queue.cc:260] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.814003 19286 raft_consensus.cc:740] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.736447 19163 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
W20260709 12:14:10.812127 19501 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.056s	sys 0.137s
I20260709 12:14:10.737784 19500 consensus_queue.cc:1048] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.084s
W20260709 12:14:10.827225 19055 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.074s	sys 0.154s
I20260709 12:14:10.827066 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
W20260709 12:14:10.827644 19055 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.074s	sys 0.154s
I20260709 12:14:10.827867 19238 raft_consensus.cc:3060] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:10.819619 19286 consensus_queue.cc:260] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:10.826109 19501 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.057s	sys 0.137s
I20260709 12:14:10.836288 19343 raft_consensus.cc:493] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.837394 19368 consensus_queue.cc:1048] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:14:10.855623 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:10.856499 19162 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:10.874670 19517 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.875339 19517 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.874480 19403 raft_consensus.cc:493] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.859104 19343 raft_consensus.cc:515] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:10.890379 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:10.884495 19403 raft_consensus.cc:515] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:10.919612 19286 ts_tablet_manager.cc:1434] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.524s	user 0.008s	sys 0.004s
I20260709 12:14:10.905690 19162 raft_consensus.cc:3060] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:10.920172 19286 ts_tablet_manager.cc:1467] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: Tablet startup took 524ms
I20260709 12:14:10.927810 19283 tablet_bootstrap.cc:492] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:10.928696 19283 ts_tablet_manager.cc:1403] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.358s	user 0.025s	sys 0.040s
I20260709 12:14:10.946957 19285 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:10.947721 19285 ts_tablet_manager.cc:1403] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.366s	user 0.029s	sys 0.036s
W20260709 12:14:10.915411 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 71454.8us
I20260709 12:14:10.949622 19500 consensus_queue.cc:1048] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:10.951334 19285 raft_consensus.cc:359] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:10.952132 19285 raft_consensus.cc:385] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:10.952422 19285 raft_consensus.cc:740] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:10.962641 19284 ts_tablet_manager.cc:1434] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.537s	user 0.021s	sys 0.006s
I20260709 12:14:10.956790 19285 consensus_queue.cc:260] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:10.924357 19286 ts_tablet_manager.cc:1469] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: Trace:
0709 12:14:06.156965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.162296 (+  5331us) service_pool.cc:224] Handling call
0709 12:14:06.165269 (+  2973us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:14:06.165423 (+   154us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:14:06.188287 (+ 22864us) tablet_metadata.cc:665] Metadata flushed
0709 12:14:06.331427 (+143140us) tablet_replica.cc:206] Creating consensus instance
0709 12:14:06.331956 (+   529us) inbound_call.cc:177] Queueing success response
0709 12:14:10.108707 (+3776751us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:14:10.395526 (+286819us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:14:10.636917 (+241391us) tablet_replica.cc:261] Starting instrumentation
0709 12:14:10.637668 (+   751us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":11969,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":12483,"tablet-open.queue_time_us":3776853}
W20260709 12:14:10.966284 19284 ts_tablet_manager.cc:1467] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Tablet startup took 537ms
I20260709 12:14:10.969476 19286 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:10.973721 19285 ts_tablet_manager.cc:1434] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:14:10.949653 19203 leader_election.cc:304] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:10.978349 19517 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:10.983839 19403 leader_election.cc:290] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:10.995746 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:10.996899 19088 raft_consensus.cc:2468] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:10.962527 19434 consensus_queue.cc:1048] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:14:11.001956 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
W20260709 12:14:11.006510 19513 log.cc:927] Time spent T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: Append to log took a long time: real 0.197s	user 0.000s	sys 0.005s
I20260709 12:14:11.011662 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.012316 19285 tablet_bootstrap.cc:492] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.012487 19239 raft_consensus.cc:2468] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
W20260709 12:14:11.015799 19130 leader_election.cc:343] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.019896 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
W20260709 12:14:11.021258 19130 leader_election.cc:343] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:10.962949 19055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 55121.9us
I20260709 12:14:11.022855 19504 consensus_queue.cc:1048] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:11.011881 19343 leader_election.cc:290] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:11.033481 19162 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
W20260709 12:14:10.969172 19284 ts_tablet_manager.cc:1469] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Trace:
0709 12:14:06.177717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.200540 (+ 22823us) service_pool.cc:224] Handling call
0709 12:14:06.225005 (+ 24465us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:14:06.225138 (+   133us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:14:06.344171 (+119033us) tablet_metadata.cc:665] Metadata flushed
0709 12:14:06.401779 (+ 57608us) tablet_replica.cc:206] Creating consensus instance
0709 12:14:06.402524 (+   745us) inbound_call.cc:177] Queueing success response
0709 12:14:10.134918 (+3732394us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:14:10.425735 (+290817us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:14:10.426525 (+   790us) tablet_replica.cc:261] Starting instrumentation
0709 12:14:10.453652 (+ 27127us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":1280,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":3163,"tablet-open.queue_time_us":3732543,"thread_start_us":108391,"threads_started":1}
I20260709 12:14:11.010636 19283 raft_consensus.cc:359] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.040663 19283 raft_consensus.cc:385] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.040848 19284 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:11.041035 19283 raft_consensus.cc:740] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.038798 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e5cc9ac6c94ccda322218e1f7eb2cb" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.045230 19283 consensus_queue.cc:260] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.076016 19055 leader_election.cc:304] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.065428 18980 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:11.062623 19284 tablet_bootstrap.cc:654] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.087023 19434 raft_consensus.cc:2804] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.094749 19434 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.095225 19434 raft_consensus.cc:3060] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.098390 19283 ts_tablet_manager.cc:1434] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.169s	user 0.008s	sys 0.003s
I20260709 12:14:11.080303 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.099733 19283 tablet_bootstrap.cc:492] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:11.099689 19162 raft_consensus.cc:2468] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:11.084616 19524 raft_consensus.cc:2804] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.101222 19524 raft_consensus.cc:697] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:11.103605 19285 tablet_bootstrap.cc:654] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.086802 19293 consensus_queue.cc:1048] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:14:11.076749 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.117568 19286 tablet_bootstrap.cc:654] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:11.085296 19130 leader_election.cc:343] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.083000 19521 consensus_queue.cc:1048] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:14:11.107510 19238 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:11.102240 19524 consensus_queue.cc:237] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.124632 19392 raft_consensus.cc:2764] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:14:11.117801 19055 leader_election.cc:304] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.123519 19130 leader_election.cc:304] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
W20260709 12:14:11.132244 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 47000.6us
I20260709 12:14:11.134280 19501 raft_consensus.cc:2749] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.145391 19293 raft_consensus.cc:2804] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.146029 19293 raft_consensus.cc:493] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.146420 19293 raft_consensus.cc:3060] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.146559 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:11.152730 19130 leader_election.cc:343] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.153383 19130 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:11.159052 19521 raft_consensus.cc:493] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.161115 19521 raft_consensus.cc:515] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.165624 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:11.171993 18912 catalog_manager.cc:5697] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:11.185720 19283 tablet_bootstrap.cc:654] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.166519 19501 raft_consensus.cc:2749] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.166097 19521 leader_election.cc:290] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:11.197045 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.198477 19239 raft_consensus.cc:2468] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:11.201674 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.204873 19165 raft_consensus.cc:2468] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:11.207998 19203 leader_election.cc:304] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.213088 19392 raft_consensus.cc:2804] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.218097 19285 tablet_bootstrap.cc:492] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:11.218762 19285 ts_tablet_manager.cc:1403] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.244s	user 0.012s	sys 0.008s
I20260709 12:14:11.222385 19434 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.223011 19285 raft_consensus.cc:359] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.223771 19285 raft_consensus.cc:385] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.224097 19285 raft_consensus.cc:740] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.224942 19285 consensus_queue.cc:260] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.226675 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:11.227162 19285 ts_tablet_manager.cc:1434] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:14:11.227487 19239 raft_consensus.cc:3060] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.228379 19285 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.236392 19285 tablet_bootstrap.cc:654] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.236869 19434 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:11.238485 19239 raft_consensus.cc:2468] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:11.213546 19392 raft_consensus.cc:493] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.256512 19285 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:11.257236 19285 ts_tablet_manager.cc:1403] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.008s
W20260709 12:14:11.251984 19129 leader_election.cc:343] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.257987 19129 leader_election.cc:304] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:11.259474 19343 raft_consensus.cc:2749] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.249122 19392 raft_consensus.cc:3060] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.265228 19283 tablet_bootstrap.cc:492] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:11.265861 19283 ts_tablet_manager.cc:1403] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.166s	user 0.009s	sys 0.006s
I20260709 12:14:11.274384 18980 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:11.275626 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:11.276409 19434 raft_consensus.cc:2804] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.276448 19091 raft_consensus.cc:3060] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.277012 19434 raft_consensus.cc:697] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:11.282608 19285 raft_consensus.cc:359] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.283844 19507 consensus_queue.cc:1048] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:11.283536 19285 raft_consensus.cc:385] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.288125 19285 raft_consensus.cc:740] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.279563 19434 consensus_queue.cc:237] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.288975 19285 consensus_queue.cc:260] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.292052 19293 raft_consensus.cc:515] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.315987 19090 raft_consensus.cc:2393] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:11.321308 19293 leader_election.cc:290] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:11.324028 19283 raft_consensus.cc:359] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.324769 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.324978 19283 raft_consensus.cc:385] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.325378 19283 raft_consensus.cc:740] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.325457 19165 raft_consensus.cc:3060] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.326176 19283 consensus_queue.cc:260] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.329154 19283 ts_tablet_manager.cc:1434] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.063s	user 0.010s	sys 0.000s
I20260709 12:14:11.330328 19283 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:11.332381 19285 ts_tablet_manager.cc:1434] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.000s
I20260709 12:14:11.335280 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:11.335741 19285 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.336946 19165 raft_consensus.cc:2468] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:11.338387 19055 leader_election.cc:304] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.339435 19293 raft_consensus.cc:2804] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.339960 19293 raft_consensus.cc:697] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:11.340907 19293 consensus_queue.cc:237] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.341982 19286 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:11.343570 19284 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:11.342530 19286 ts_tablet_manager.cc:1403] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.373s	user 0.016s	sys 0.022s
I20260709 12:14:11.346309 19284 ts_tablet_manager.cc:1403] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.306s	user 0.016s	sys 0.018s
I20260709 12:14:11.342176 19091 raft_consensus.cc:2468] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:11.350209 19507 consensus_queue.cc:1048] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:11.350484 19286 raft_consensus.cc:359] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.351312 19286 raft_consensus.cc:385] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.351811 19286 raft_consensus.cc:740] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.357833 19283 tablet_bootstrap.cc:654] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.358971 19286 consensus_queue.cc:260] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.360539 19392 raft_consensus.cc:515] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.364666 19286 ts_tablet_manager.cc:1434] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:14:11.366012 19286 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:11.368547 19285 tablet_bootstrap.cc:654] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.371417 18911 catalog_manager.cc:5697] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:11.376747 19286 tablet_bootstrap.cc:654] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.384037 19239 raft_consensus.cc:2393] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:11.401450 19283 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:11.402077 19283 ts_tablet_manager.cc:1403] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.010s
I20260709 12:14:11.407193 19505 raft_consensus.cc:493] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.407883 19505 raft_consensus.cc:515] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.410405 19505 leader_election.cc:290] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:11.417309 19284 raft_consensus.cc:359] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.419925 19284 raft_consensus.cc:385] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.420284 19284 raft_consensus.cc:740] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.426872 19284 consensus_queue.cc:260] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.430799 19284 ts_tablet_manager.cc:1434] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 12:14:11.431560 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:11.435286 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.436156 19165 raft_consensus.cc:2468] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:11.437273 19284 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:11.437980 19055 leader_election.cc:304] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.438057 18911 catalog_manager.cc:5697] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:11.439783 19526 raft_consensus.cc:2804] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.440263 19526 raft_consensus.cc:493] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.440681 19526 raft_consensus.cc:3060] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.441077 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:11.443118 19055 leader_election.cc:343] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.447595 19392 leader_election.cc:290] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:11.447759 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.448616 19165 raft_consensus.cc:2393] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:11.449875 19203 leader_election.cc:304] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:11.452014 19392 raft_consensus.cc:2749] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.454288 19283 raft_consensus.cc:359] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.457619 18912 catalog_manager.cc:5697] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:11.463744 19285 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:11.470011 19285 ts_tablet_manager.cc:1403] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.135s	user 0.014s	sys 0.003s
I20260709 12:14:11.471634 19284 tablet_bootstrap.cc:654] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.468061 19526 raft_consensus.cc:515] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.459201 19283 raft_consensus.cc:385] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.467510 19286 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:11.474668 19283 raft_consensus.cc:740] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.475955 19526 leader_election.cc:290] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:11.475771 19283 consensus_queue.cc:260] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.476675 19286 ts_tablet_manager.cc:1403] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.111s	user 0.013s	sys 0.003s
I20260709 12:14:11.478183 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.478878 19283 ts_tablet_manager.cc:1434] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.004s
I20260709 12:14:11.478821 19165 raft_consensus.cc:3060] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.479120 19285 raft_consensus.cc:359] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.479952 19285 raft_consensus.cc:385] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.480185 19283 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:11.480209 19285 raft_consensus.cc:740] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.476526 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:11.481356 19285 consensus_queue.cc:260] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.481766 19286 raft_consensus.cc:359] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.482563 19286 raft_consensus.cc:385] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:14:11.482609 19055 leader_election.cc:343] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.482926 19286 raft_consensus.cc:740] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.484127 19285 ts_tablet_manager.cc:1434] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:14:11.483749 19286 consensus_queue.cc:260] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.485236 19285 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.486481 19286 ts_tablet_manager.cc:1434] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:14:11.488281 19286 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:11.489526 19165 raft_consensus.cc:2468] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:11.492589 19285 tablet_bootstrap.cc:654] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.493973 19055 leader_election.cc:304] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:11.494864 19526 raft_consensus.cc:2804] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.495389 19526 raft_consensus.cc:697] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:11.496351 19526 consensus_queue.cc:237] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.502149 19434 raft_consensus.cc:493] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.502703 19434 raft_consensus.cc:515] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.511888 19434 leader_election.cc:290] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:11.512854 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.513815 19239 raft_consensus.cc:2468] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:11.515463 18980 leader_election.cc:304] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:11.516443 19434 raft_consensus.cc:2804] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.516878 19434 raft_consensus.cc:493] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.517275 19434 raft_consensus.cc:3060] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.513000 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.520884 19165 raft_consensus.cc:2468] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:11.522387 19283 tablet_bootstrap.cc:654] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.534137 19286 tablet_bootstrap.cc:654] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.539042 19434 raft_consensus.cc:515] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.545040 19285 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:11.545040 19284 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:11.547096 19284 ts_tablet_manager.cc:1403] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.112s	user 0.014s	sys 0.006s
I20260709 12:14:11.546788 18913 catalog_manager.cc:5697] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:11.548832 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.549507 19165 raft_consensus.cc:3060] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.551079 19434 leader_election.cc:290] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:11.550976 19284 raft_consensus.cc:359] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.551877 19284 raft_consensus.cc:385] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.551982 19285 ts_tablet_manager.cc:1403] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.000s
I20260709 12:14:11.552232 19284 raft_consensus.cc:740] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.555646 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:11.556368 19239 raft_consensus.cc:3060] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.553165 19284 consensus_queue.cc:260] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.556531 19286 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:11.557626 19286 ts_tablet_manager.cc:1403] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.005s
I20260709 12:14:11.557714 19285 raft_consensus.cc:359] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.558475 19285 raft_consensus.cc:385] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.558759 19285 raft_consensus.cc:740] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.559742 19285 consensus_queue.cc:260] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.562247 19285 ts_tablet_manager.cc:1434] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 12:14:11.563314 19285 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.563683 19526 consensus_queue.cc:1048] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:11.567446 19239 raft_consensus.cc:2468] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:11.569398 19286 raft_consensus.cc:359] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.570251 18980 leader_election.cc:304] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:11.573465 19286 raft_consensus.cc:385] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.573803 19286 raft_consensus.cc:740] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.574232 19285 tablet_bootstrap.cc:654] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.574713 19286 consensus_queue.cc:260] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.571641 19284 ts_tablet_manager.cc:1434] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:14:11.576788 19284 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:11.576949 19526 raft_consensus.cc:493] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.577503 19357 raft_consensus.cc:2804] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.577430 19526 raft_consensus.cc:515] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.578274 19357 raft_consensus.cc:697] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:11.579365 19357 consensus_queue.cc:237] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.579857 19526 leader_election.cc:290] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.580255 19286 ts_tablet_manager.cc:1434] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:14:11.584481 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5373fb94420c4369bab88209ddbd5324" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:11.586127 19055 leader_election.cc:343] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.586498 19286 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:11.587182 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5373fb94420c4369bab88209ddbd5324" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.588095 19237 raft_consensus.cc:2468] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:11.589684 19056 leader_election.cc:304] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:11.590590 19526 raft_consensus.cc:2804] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.591087 19526 raft_consensus.cc:493] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.591442 19526 raft_consensus.cc:3060] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.592809 19165 raft_consensus.cc:2468] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:11.594790 19284 tablet_bootstrap.cc:654] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.596423 19293 raft_consensus.cc:493] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.597007 19293 raft_consensus.cc:515] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.599568 19293 leader_election.cc:290] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.600071 19286 tablet_bootstrap.cc:654] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.602653 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:11.618620 19055 leader_election.cc:343] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.621554 19283 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:11.622140 19343 raft_consensus.cc:493] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.622263 19283 ts_tablet_manager.cc:1403] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.004s
I20260709 12:14:11.623049 19343 raft_consensus.cc:515] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.628427 19283 raft_consensus.cc:359] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.629499 19283 raft_consensus.cc:385] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.629834 19283 raft_consensus.cc:740] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.629933 19293 consensus_queue.cc:1048] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:11.635864 19343 leader_election.cc:290] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.590241 19285 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:11.641068 19285 ts_tablet_manager.cc:1403] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.009s
I20260709 12:14:11.643265 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:11.613747 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.637192 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.641005 19293 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.644563 19285 raft_consensus.cc:359] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.645362 19285 raft_consensus.cc:385] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.645699 19285 raft_consensus.cc:740] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.611951 19526 raft_consensus.cc:515] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:11.646404 19056 leader_election.cc:343] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:11.646792 19130 leader_election.cc:343] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.648669 19056 leader_election.cc:304] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:11.650118 19522 raft_consensus.cc:2749] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.647008 19285 consensus_queue.cc:260] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.652755 19285 ts_tablet_manager.cc:1434] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:14:11.654924 19526 leader_election.cc:290] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:11.659776 19129 leader_election.cc:343] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.660358 19129 leader_election.cc:304] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:11.653841 19285 tablet_bootstrap.cc:492] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.655855 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5373fb94420c4369bab88209ddbd5324" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:11.663794 19238 raft_consensus.cc:3060] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.667443 19283 consensus_queue.cc:260] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.656575 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5373fb94420c4369bab88209ddbd5324" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:11.645089 19293 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.673228 19284 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:11.673920 19284 ts_tablet_manager.cc:1403] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.006s
I20260709 12:14:11.675407 19392 raft_consensus.cc:493] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.676075 19392 raft_consensus.cc:515] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.677423 19286 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:11.678047 19286 ts_tablet_manager.cc:1403] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.006s
W20260709 12:14:11.679112 19055 leader_election.cc:343] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.679495 19392 leader_election.cc:290] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.680747 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.681543 19284 raft_consensus.cc:359] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.682320 19284 raft_consensus.cc:385] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.682629 19284 raft_consensus.cc:740] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
W20260709 12:14:11.682874 19056 leader_election.cc:343] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.682940 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e5cc9ac6c94ccda322218e1f7eb2cb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:11.683444 19284 consensus_queue.cc:260] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.684317 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e5cc9ac6c94ccda322218e1f7eb2cb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.684754 19014 raft_consensus.cc:2468] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:11.685102 19164 raft_consensus.cc:2468] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:11.685534 19284 ts_tablet_manager.cc:1434] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.005s
I20260709 12:14:11.686666 19284 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:11.686491 19203 leader_election.cc:304] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.687539 19285 tablet_bootstrap.cc:654] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.688051 19368 raft_consensus.cc:2804] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.688486 19368 raft_consensus.cc:493] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.688860 19368 raft_consensus.cc:3060] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.689658 19293 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:11.690322 19343 raft_consensus.cc:2749] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.695542 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.695721 19286 raft_consensus.cc:359] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.700588 19286 raft_consensus.cc:385] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.700987 19286 raft_consensus.cc:740] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.691848 19283 ts_tablet_manager.cc:1434] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.067s	user 0.006s	sys 0.005s
I20260709 12:14:11.701823 19286 consensus_queue.cc:260] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.730378 19392 raft_consensus.cc:493] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.730903 19392 raft_consensus.cc:515] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.702332 18909 catalog_manager.cc:5697] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:11.709677 19502 raft_consensus.cc:493] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.718860 19283 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:11.737627 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.737752 19286 ts_tablet_manager.cc:1434] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.005s
I20260709 12:14:11.738420 19165 raft_consensus.cc:2468] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:11.720471 19238 raft_consensus.cc:2468] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:11.707078 19164 raft_consensus.cc:2468] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:11.737828 19502 raft_consensus.cc:515] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.741075 19502 leader_election.cc:290] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:11.739125 19286 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:11.742004 19055 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:11.743060 19293 raft_consensus.cc:2804] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.743470 19293 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.743808 19293 raft_consensus.cc:3060] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.749423 19286 tablet_bootstrap.cc:654] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.753057 19203 leader_election.cc:304] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.754613 19543 raft_consensus.cc:2804] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.755160 19543 raft_consensus.cc:493] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.752995 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0ead3da3436d97931d4aff80a357" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:11.755674 19543 raft_consensus.cc:3060] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.756248 19293 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:11.757191 19130 leader_election.cc:343] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.726738 19284 tablet_bootstrap.cc:654] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.754194 19283 tablet_bootstrap.cc:654] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.765157 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:11.767374 19203 leader_election.cc:343] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.769097 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0ead3da3436d97931d4aff80a357" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:11.769585 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.770498 19164 raft_consensus.cc:3060] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.770841 19056 leader_election.cc:304] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:11.772063 19526 raft_consensus.cc:2804] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.772589 19526 raft_consensus.cc:697] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
W20260709 12:14:11.772981 19130 leader_election.cc:343] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.773509 19130 leader_election.cc:304] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:11.773586 19526 consensus_queue.cc:237] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.762967 19392 leader_election.cc:290] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.774461 19502 raft_consensus.cc:2749] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.787851 19293 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:11.789245 19285 tablet_bootstrap.cc:492] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:11.798472 19285 ts_tablet_manager.cc:1403] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.145s	user 0.023s	sys 0.005s
I20260709 12:14:11.803385 19502 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.804055 19502 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.806627 19502 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:11.791424 19543 raft_consensus.cc:515] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:11.804386 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:11.790546 19368 raft_consensus.cc:515] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:11.810156 19286 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:11.810613 19368 leader_election.cc:290] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.810786 19286 ts_tablet_manager.cc:1403] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.072s	user 0.025s	sys 0.020s
I20260709 12:14:11.812983 19285 raft_consensus.cc:359] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.821125 19285 raft_consensus.cc:385] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.821477 19285 raft_consensus.cc:740] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.819952 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:11.829783 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e5cc9ac6c94ccda322218e1f7eb2cb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:11.830788 19015 raft_consensus.cc:3060] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.824106 19357 consensus_queue.cc:1048] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:14:11.817319 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:11.835170 19129 leader_election.cc:343] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.836740 19434 raft_consensus.cc:493] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.837314 19434 raft_consensus.cc:515] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.839274 19286 raft_consensus.cc:359] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.840024 19286 raft_consensus.cc:385] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.840319 19286 raft_consensus.cc:740] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.841039 19286 consensus_queue.cc:260] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.842787 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:11.842809 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55dd90070791474ea046bd86b13e3f75" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.844378 19434 leader_election.cc:290] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:11.844674 19165 raft_consensus.cc:2393] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:11.834026 19285 consensus_queue.cc:260] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.848117 19286 ts_tablet_manager.cc:1434] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:14:11.851454 19286 tablet_bootstrap.cc:492] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:11.854954 19285 ts_tablet_manager.cc:1434] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.056s	user 0.004s	sys 0.004s
I20260709 12:14:11.858165 19285 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:11.854158 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e5cc9ac6c94ccda322218e1f7eb2cb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.860266 19165 raft_consensus.cc:3060] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.858100 18980 leader_election.cc:304] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:11.863178 19164 raft_consensus.cc:2468] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:11.866391 19434 raft_consensus.cc:3060] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.873849 19543 leader_election.cc:290] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:11.877436 19283 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:11.876353 19055 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.877803 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:11.878258 19283 ts_tablet_manager.cc:1403] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.176s	user 0.015s	sys 0.000s
I20260709 12:14:11.876159 19165 raft_consensus.cc:2468] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:11.879136 19293 raft_consensus.cc:2804] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:14:11.879730 19203 leader_election.cc:343] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.879777 19293 raft_consensus.cc:697] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:11.881417 19293 consensus_queue.cc:237] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.883584 19283 raft_consensus.cc:359] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.884388 19283 raft_consensus.cc:385] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.884699 19283 raft_consensus.cc:740] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.885673 19283 consensus_queue.cc:260] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:11.887413 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:11.888127 19283 ts_tablet_manager.cc:1434] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:14:11.889343 19283 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:11.888152 19163 raft_consensus.cc:3060] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.892277 19562 raft_consensus.cc:493] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.892868 19562 raft_consensus.cc:515] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:11.882741 19015 raft_consensus.cc:2468] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:11.903925 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:11.899919 19562 leader_election.cc:290] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
W20260709 12:14:11.912775 19056 leader_election.cc:343] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.900185 19203 leader_election.cc:304] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:11.924504 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0ead3da3436d97931d4aff80a357" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:11.925211 19130 leader_election.cc:343] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.925549 19543 raft_consensus.cc:2804] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:11.925823 19130 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:11.926190 19543 raft_consensus.cc:697] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:11.927066 19283 tablet_bootstrap.cc:654] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.927244 19543 consensus_queue.cc:237] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:11.929064 19056 leader_election.cc:343] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.929926 19502 raft_consensus.cc:2749] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.941226 19286 tablet_bootstrap.cc:654] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.938292 19562 consensus_queue.cc:1048] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:14:11.947633 19521 raft_consensus.cc:493] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.948513 19285 tablet_bootstrap.cc:654] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:11.948264 19521 raft_consensus.cc:515] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.948972 19284 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:11.952898 19284 ts_tablet_manager.cc:1403] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.266s	user 0.032s	sys 0.012s
I20260709 12:14:11.958534 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:11.959378 19089 raft_consensus.cc:2468] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:11.960857 19204 leader_election.cc:304] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:11.961241 19284 raft_consensus.cc:359] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.961952 19392 raft_consensus.cc:2804] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:11.962390 19284 raft_consensus.cc:385] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:11.962728 19392 raft_consensus.cc:493] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.962914 19284 raft_consensus.cc:740] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:11.963275 19392 raft_consensus.cc:3060] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:11.963999 19284 consensus_queue.cc:260] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:11.966228 19434 raft_consensus.cc:2749] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:11.967067 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:11.973047 19526 consensus_queue.cc:1048] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:11.968026 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0ead3da3436d97931d4aff80a357" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:11.974207 19165 raft_consensus.cc:2468] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:11.955221 19521 leader_election.cc:290] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:11.934263 18910 catalog_manager.cc:5697] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:11.979903 19203 leader_election.cc:343] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:11.978664 19284 ts_tablet_manager.cc:1434] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 12:14:11.982793 19055 leader_election.cc:304] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:11.925052 19163 raft_consensus.cc:2468] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:11.997115 19203 leader_election.cc:304] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:11.997238 19357 consensus_queue.cc:1048] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:11.990811 19284 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:12.005803 19521 raft_consensus.cc:2804] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:12.009493 19521 raft_consensus.cc:697] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:12.010562 19521 consensus_queue.cc:237] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:12.019282 19285 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:12.020157 19285 ts_tablet_manager.cc:1403] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.000s
I20260709 12:14:12.021579 19526 raft_consensus.cc:2804] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:12.028558 19526 raft_consensus.cc:493] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.029999 19526 raft_consensus.cc:3060] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.036388 19293 consensus_queue.cc:1048] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:12.027685 19392 raft_consensus.cc:515] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.038046 19522 raft_consensus.cc:493] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.048914 19522 raft_consensus.cc:515] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.047724 19502 raft_consensus.cc:493] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.061735 19502 raft_consensus.cc:515] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.063899 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:12.064312 19502 leader_election.cc:290] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:12.064819 19089 raft_consensus.cc:3060] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.069252 19522 leader_election.cc:290] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:12.072908 19577 log.cc:927] Time spent T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:14:12.074191 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:12.075954 19055 leader_election.cc:343] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:12.077210 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:12.078119 19162 raft_consensus.cc:2468] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:12.079064 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:12.088363 19284 tablet_bootstrap.cc:654] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.046567 19392 leader_election.cc:290] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:12.092458 19283 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:12.091676 19285 raft_consensus.cc:359] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.091701 18908 catalog_manager.cc:5697] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:12.088406 19129 leader_election.cc:343] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:12.099355 19285 raft_consensus.cc:385] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.095535 19055 leader_election.cc:304] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:12.096979 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:12.099499 19283 ts_tablet_manager.cc:1403] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.210s	user 0.018s	sys 0.003s
I20260709 12:14:12.115724 19285 raft_consensus.cc:740] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.106393 19526 raft_consensus.cc:515] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.125058 19522 raft_consensus.cc:2804] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:12.125783 19522 raft_consensus.cc:493] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.126286 19522 raft_consensus.cc:3060] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.125988 19562 consensus_queue.cc:1048] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:12.137574 19285 consensus_queue.cc:260] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
W20260709 12:14:12.150501 19203 leader_election.cc:343] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:12.153993 19286 tablet_bootstrap.cc:492] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:12.154738 19286 ts_tablet_manager.cc:1403] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.304s	user 0.015s	sys 0.008s
I20260709 12:14:12.091216 19434 consensus_queue.cc:1048] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260709 12:14:12.161049 19526 leader_election.cc:290] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:12.167374 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0ead3da3436d97931d4aff80a357" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:12.168329 19162 raft_consensus.cc:3060] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.168085 19286 raft_consensus.cc:359] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.169912 19089 raft_consensus.cc:2468] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:12.169359 18908 catalog_manager.cc:5697] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:12.171367 19204 leader_election.cc:304] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:12.171208 19434 consensus_queue.cc:1048] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:12.172727 19392 raft_consensus.cc:2804] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:12.173040 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24dc0ead3da3436d97931d4aff80a357" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:12.168951 19286 raft_consensus.cc:385] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.175932 19285 ts_tablet_manager.cc:1434] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.155s	user 0.011s	sys 0.000s
I20260709 12:14:12.178750 19285 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:12.175657 19286 raft_consensus.cc:740] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.184379 19286 consensus_queue.cc:260] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.175068 19392 raft_consensus.cc:697] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
W20260709 12:14:12.175324 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 140297us
I20260709 12:14:12.195971 19237 raft_consensus.cc:3060] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.225658 19526 consensus_queue.cc:1048] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:14:12.237306 19286 ts_tablet_manager.cc:1434] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.001s
I20260709 12:14:12.239742 19286 tablet_bootstrap.cc:492] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:12.195097 19522 raft_consensus.cc:515] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.249504 19522 leader_election.cc:290] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:12.251621 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:12.254588 19283 raft_consensus.cc:359] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.256273 19283 raft_consensus.cc:385] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.270871 19089 raft_consensus.cc:2393] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:12.275879 19283 raft_consensus.cc:740] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.210767 19392 consensus_queue.cc:237] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.322291 19284 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:12.323026 19284 ts_tablet_manager.cc:1403] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.332s	user 0.017s	sys 0.010s
I20260709 12:14:12.331012 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:12.331830 19165 raft_consensus.cc:3060] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.337956 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:12.339725 19284 raft_consensus.cc:359] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.341933 19284 raft_consensus.cc:385] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.342330 19284 raft_consensus.cc:740] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.343225 19284 consensus_queue.cc:260] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.345706 19130 leader_election.cc:304] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:12.347793 19284 ts_tablet_manager.cc:1434] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
W20260709 12:14:12.354622 19056 consensus_peers.cc:597] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Couldn't send request to peer 2c0b3983ae6c48108f6f3632116cf5a8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:12.360505 19284 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:12.329099 19283 consensus_queue.cc:260] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.369792 19284 tablet_bootstrap.cc:654] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:12.376363 19589 log.cc:927] Time spent T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:12.377627 18912 catalog_manager.cc:5697] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:12.370373 19283 ts_tablet_manager.cc:1434] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.252s	user 0.007s	sys 0.002s
I20260709 12:14:12.384023 19286 tablet_bootstrap.cc:654] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.381692 19283 tablet_bootstrap.cc:492] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:12.388770 19285 tablet_bootstrap.cc:654] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:12.389190 19055 leader_election.cc:343] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:12.392434 19237 raft_consensus.cc:2468] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:12.397322 19165 raft_consensus.cc:2468] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:12.400975 19013 raft_consensus.cc:3060] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.407688 19343 raft_consensus.cc:2749] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:12.379438 19162 raft_consensus.cc:2468] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:12.422730 19055 leader_election.cc:304] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:12.432600 19522 raft_consensus.cc:2804] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:12.433157 19522 raft_consensus.cc:697] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:12.433230 19055 leader_election.cc:304] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:12.434253 19562 raft_consensus.cc:2804] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:12.434156 19522 consensus_queue.cc:237] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.434837 19562 raft_consensus.cc:697] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:12.436025 19562 consensus_queue.cc:237] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.426399 19286 tablet_bootstrap.cc:492] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:12.438256 19286 ts_tablet_manager.cc:1403] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.199s	user 0.014s	sys 0.004s
I20260709 12:14:12.446199 19595 raft_consensus.cc:493] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.451400 19595 raft_consensus.cc:515] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.454028 19595 leader_election.cc:290] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:12.457180 19524 consensus_queue.cc:1048] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:12.459874 19285 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:12.463367 19285 ts_tablet_manager.cc:1403] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.285s	user 0.021s	sys 0.005s
I20260709 12:14:12.462291 19343 raft_consensus.cc:493] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.465080 19343 raft_consensus.cc:515] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.465947 19235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5373fb94420c4369bab88209ddbd5324" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:12.469156 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5373fb94420c4369bab88209ddbd5324" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:12.471880 19286 raft_consensus.cc:359] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.472560 19286 raft_consensus.cc:385] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.472811 19286 raft_consensus.cc:740] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.474517 19283 tablet_bootstrap.cc:654] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.480315 19505 raft_consensus.cc:493] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.484637 19505 raft_consensus.cc:515] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.488441 19521 consensus_queue.cc:1048] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:14:12.495162 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:12.497834 18980 leader_election.cc:304] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:12.499948 19286 consensus_queue.cc:260] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.516165 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:12.511327 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:12.522264 19505 leader_election.cc:290] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:12.517799 19343 leader_election.cc:290] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:12.520144 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:12.551812 19284 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:12.552346 19286 ts_tablet_manager.cc:1434] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.114s	user 0.005s	sys 0.004s
I20260709 12:14:12.552692 19284 ts_tablet_manager.cc:1403] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.192s	user 0.027s	sys 0.007s
W20260709 12:14:12.519644 19129 leader_election.cc:343] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:12.553680 19286 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:12.557243 19090 raft_consensus.cc:2468] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
W20260709 12:14:12.560011 19591 log.cc:927] Time spent T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:12.563992 19285 raft_consensus.cc:359] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.564881 19285 raft_consensus.cc:385] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.570770 18910 catalog_manager.cc:5697] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:12.544865 19521 consensus_queue.cc:1048] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:14:12.577226 19130 leader_election.cc:304] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:12.579092 19581 raft_consensus.cc:2804] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:12.579533 19581 raft_consensus.cc:493] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.579933 19581 raft_consensus.cc:3060] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.586095 19392 consensus_queue.cc:1048] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:12.565222 19285 raft_consensus.cc:740] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.595443 19285 consensus_queue.cc:260] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.596581 19284 raft_consensus.cc:359] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:12.599337 19284 raft_consensus.cc:385] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.599785 19284 raft_consensus.cc:740] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.598323 19434 raft_consensus.cc:2749] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:12.550904 19055 leader_election.cc:343] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:12.609810 19055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 61607.4us
I20260709 12:14:12.611142 19284 consensus_queue.cc:260] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:12.613127 19284 ts_tablet_manager.cc:1434] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.001s
I20260709 12:14:12.614233 19284 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:12.616211 19505 consensus_queue.cc:1048] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:12.621030 19286 tablet_bootstrap.cc:654] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.629216 19284 tablet_bootstrap.cc:654] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.636584 18908 catalog_manager.cc:5697] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:12.644826 19285 ts_tablet_manager.cc:1434] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.181s	user 0.007s	sys 0.003s
I20260709 12:14:12.638857 18908 catalog_manager.cc:5697] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:12.646441 19285 tablet_bootstrap.cc:492] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:12.628803 19581 raft_consensus.cc:515] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.660684 19581 leader_election.cc:290] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:12.661953 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:12.662714 19091 raft_consensus.cc:3060] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.655629 19165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:12.668145 19165 raft_consensus.cc:2393] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
I20260709 12:14:12.670766 19521 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.671433 19521 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.674122 19055 leader_election.cc:304] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:12.675717 19091 raft_consensus.cc:2468] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:12.676379 19522 raft_consensus.cc:2749] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:12.676957 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:12.678339 19161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:12.678802 19130 leader_election.cc:304] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:12.679454 19161 raft_consensus.cc:2393] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:12.682303 19521 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:12.683681 19203 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:12.695216 19521 raft_consensus.cc:3060] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.702677 19285 tablet_bootstrap.cc:654] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.715060 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:12.697299 19581 raft_consensus.cc:2804] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:12.723620 19283 tablet_bootstrap.cc:492] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:12.724299 19283 ts_tablet_manager.cc:1403] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.343s	user 0.051s	sys 0.041s
I20260709 12:14:12.724793 19434 raft_consensus.cc:493] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.725368 19434 raft_consensus.cc:515] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:12.728865 19596 raft_consensus.cc:493] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.729445 19596 raft_consensus.cc:515] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:12.728312 19129 leader_election.cc:343] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:12.728233 19283 raft_consensus.cc:359] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.731809 19283 raft_consensus.cc:385] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.729723 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:12.732518 19283 raft_consensus.cc:740] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.723575 19581 raft_consensus.cc:697] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:12.744067 19283 consensus_queue.cc:260] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.746112 19283 ts_tablet_manager.cc:1434] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:14:12.747638 19596 leader_election.cc:290] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:12.750842 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:12.751362 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:12.752256 19091 raft_consensus.cc:2393] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:12.753311 18980 leader_election.cc:304] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:12.754892 19596 raft_consensus.cc:3060] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:12.755388 19283 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:12.757326 19581 consensus_queue.cc:237] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.732280 19434 leader_election.cc:290] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:12.763551 19521 raft_consensus.cc:2749] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:12.787531 19285 tablet_bootstrap.cc:492] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:12.788475 19285 ts_tablet_manager.cc:1403] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.142s	user 0.013s	sys 0.005s
I20260709 12:14:12.792681 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecdce0779954a328062dfca1a0364e4" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:12.799604 19596 raft_consensus.cc:2749] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:12.869118 19581 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.870107 19581 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.871865 19286 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:12.872522 19286 ts_tablet_manager.cc:1403] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.319s	user 0.026s	sys 0.014s
I20260709 12:14:12.876204 19286 raft_consensus.cc:359] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:12.876984 19286 raft_consensus.cc:385] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.878258 19286 raft_consensus.cc:740] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.879042 19286 consensus_queue.cc:260] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:12.885713 19286 ts_tablet_manager.cc:1434] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:14:12.886960 19286 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:12.887267 18979 leader_election.cc:304] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:12.893461 19393 raft_consensus.cc:2749] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:12.894086 19286 tablet_bootstrap.cc:654] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:12.897035 18980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 82571.2us
I20260709 12:14:12.903151 19522 consensus_queue.cc:1048] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:12.906185 19286 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:12.906802 19286 ts_tablet_manager.cc:1403] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.010s
I20260709 12:14:12.914417 19284 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:12.915256 19284 ts_tablet_manager.cc:1403] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.301s	user 0.042s	sys 0.044s
I20260709 12:14:12.917855 19284 raft_consensus.cc:359] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.918592 19284 raft_consensus.cc:385] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.908561 19581 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:12.918900 19284 raft_consensus.cc:740] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.924278 19284 consensus_queue.cc:260] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:12.926949 19284 ts_tablet_manager.cc:1434] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:14:12.927276 19283 tablet_bootstrap.cc:654] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:12.929800 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:12.936836 19286 raft_consensus.cc:359] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.937683 19286 raft_consensus.cc:385] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.938027 19286 raft_consensus.cc:740] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.938836 19286 consensus_queue.cc:260] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:12.941138 19286 ts_tablet_manager.cc:1434] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:14:12.932482 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:12.932209 19578 consensus_queue.cc:1048] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:12.951251 19368 consensus_queue.cc:1048] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:12.884735 19285 raft_consensus.cc:359] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.953410 19285 raft_consensus.cc:385] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:12.953720 19285 raft_consensus.cc:740] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:12.963948 19285 consensus_queue.cc:260] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:12.980157 19090 raft_consensus.cc:2468] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
W20260709 12:14:12.985487 19130 leader_election.cc:343] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.010092 19285 ts_tablet_manager.cc:1434] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.221s	user 0.012s	sys 0.003s
I20260709 12:14:13.011512 19285 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:13.020704 19284 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:13.030807 19055 consensus_peers.cc:597] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:13.038141 19286 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:13.019991 19578 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.046883 19595 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.049912 19595 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
W20260709 12:14:13.067171 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:13.073994 19562 consensus_queue.cc:1048] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:14:13.082448 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:13.083410 19162 raft_consensus.cc:2468] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:13.085070 18979 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:13.090718 19434 raft_consensus.cc:2804] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:13.091164 19434 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.091538 19434 raft_consensus.cc:3060] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.106642 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:13.107475 19090 raft_consensus.cc:2468] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:13.071409 19595 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:13.117821 19284 tablet_bootstrap.cc:654] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.104753 18911 catalog_manager.cc:5697] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:13.048748 19578 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.123585 19130 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:13.125211 19578 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:13.130736 19286 tablet_bootstrap.cc:654] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.133188 19604 raft_consensus.cc:2804] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:13.150259 19283 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:13.151165 19283 ts_tablet_manager.cc:1403] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.396s	user 0.084s	sys 0.104s
I20260709 12:14:13.152694 19604 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.153153 19604 raft_consensus.cc:3060] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.156556 19283 raft_consensus.cc:359] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.184239 19283 raft_consensus.cc:385] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.184751 19283 raft_consensus.cc:740] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.185936 19283 consensus_queue.cc:260] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.189574 19283 ts_tablet_manager.cc:1434] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.010s
I20260709 12:14:13.190193 19285 tablet_bootstrap.cc:654] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.191641 19283 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
W20260709 12:14:13.142757 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 97517.2us
I20260709 12:14:13.203171 19618 raft_consensus.cc:493] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.203254 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:13.203773 19618 raft_consensus.cc:515] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.207342 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:13.208825 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718f20f68825449c836f67bd5a4bae55" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:13.236724 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718f20f68825449c836f67bd5a4bae55" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:13.236732 19204 leader_election.cc:343] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.239959 19618 leader_election.cc:290] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:13.242760 19015 raft_consensus.cc:3060] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.261875 19434 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
W20260709 12:14:13.271559 19203 leader_election.cc:343] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:13.295634 19203 leader_election.cc:343] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.296238 19203 leader_election.cc:304] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
W20260709 12:14:13.265496 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 152807us
I20260709 12:14:13.299919 19620 raft_consensus.cc:2749] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:13.308152 19286 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:13.311136 19286 ts_tablet_manager.cc:1403] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.278s	user 0.016s	sys 0.008s
I20260709 12:14:13.312887 19434 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:13.313789 19606 consensus_queue.cc:1048] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:13.314061 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:13.316133 19162 raft_consensus.cc:3060] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.317312 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
W20260709 12:14:13.317894 19203 leader_election.cc:343] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.318073 19090 raft_consensus.cc:3060] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.318490 19203 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:13.324522 19604 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.327195 19620 raft_consensus.cc:2749] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:13.335420 19604 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:13.351625 19286 raft_consensus.cc:359] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.352756 19286 raft_consensus.cc:385] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.353199 19286 raft_consensus.cc:740] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
W20260709 12:14:13.359316 19129 consensus_peers.cc:597] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:13.365737 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:13.366586 19091 raft_consensus.cc:3060] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.373659 19579 consensus_queue.cc:1048] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:13.374269 19162 raft_consensus.cc:2468] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:13.379216 19235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:13.386765 18979 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:13.392472 19434 raft_consensus.cc:2804] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:13.394831 19285 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:13.395449 19285 ts_tablet_manager.cc:1403] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.384s	user 0.019s	sys 0.015s
I20260709 12:14:13.389086 19286 consensus_queue.cc:260] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.399132 19286 ts_tablet_manager.cc:1434] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.002s
I20260709 12:14:13.399814 19284 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:13.400357 19284 ts_tablet_manager.cc:1403] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.466s	user 0.007s	sys 0.014s
I20260709 12:14:13.394842 19434 raft_consensus.cc:697] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:13.402360 19434 consensus_queue.cc:237] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.404829 19286 tablet_bootstrap.cc:492] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:13.366551 19283 tablet_bootstrap.cc:654] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.413865 19285 raft_consensus.cc:359] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.414821 19285 raft_consensus.cc:385] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.415161 19285 raft_consensus.cc:740] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.416091 19604 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.398799 19091 raft_consensus.cc:2468] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:13.416769 19604 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.416064 19285 consensus_queue.cc:260] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:13.419420 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 49221.8us
I20260709 12:14:13.418092 19130 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:13.424675 19604 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:13.424906 19579 raft_consensus.cc:2804] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:13.425463 19579 raft_consensus.cc:697] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:13.426424 19579 consensus_queue.cc:237] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.429678 19285 ts_tablet_manager.cc:1434] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:14:13.430784 19285 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:13.432076 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:13.436487 19130 leader_election.cc:343] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:13.439128 19129 leader_election.cc:343] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.458007 19284 raft_consensus.cc:359] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.462164 19284 raft_consensus.cc:385] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.462544 19284 raft_consensus.cc:740] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.466707 19286 tablet_bootstrap.cc:654] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.481076 19284 consensus_queue.cc:260] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.493152 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:13.495926 19130 leader_election.cc:343] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.496461 19130 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:13.497776 19579 raft_consensus.cc:2749] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:13.505023 19283 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:13.505726 19283 ts_tablet_manager.cc:1403] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.315s	user 0.013s	sys 0.018s
I20260709 12:14:13.512156 19283 raft_consensus.cc:359] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.513033 19283 raft_consensus.cc:385] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.513417 19283 raft_consensus.cc:740] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.512486 18911 catalog_manager.cc:5697] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:13.514124 19283 consensus_queue.cc:260] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.517443 19283 ts_tablet_manager.cc:1434] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:14:13.518471 19283 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:13.522490 19285 tablet_bootstrap.cc:654] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.526037 19283 tablet_bootstrap.cc:654] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.532251 19090 raft_consensus.cc:2468] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:13.548581 18908 catalog_manager.cc:5697] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:13.562580 19284 ts_tablet_manager.cc:1434] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.162s	user 0.005s	sys 0.003s
W20260709 12:14:13.574007 18980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 64036.7us
I20260709 12:14:13.582494 19286 tablet_bootstrap.cc:492] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:13.583190 19286 ts_tablet_manager.cc:1403] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.183s	user 0.019s	sys 0.035s
W20260709 12:14:13.583590 18979 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 67791.1us
I20260709 12:14:13.589877 19286 raft_consensus.cc:359] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.596182 19286 raft_consensus.cc:385] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.596411 19286 raft_consensus.cc:740] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.602319 19283 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:13.605801 19285 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:13.613953 19283 ts_tablet_manager.cc:1403] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.096s	user 0.026s	sys 0.030s
I20260709 12:14:13.614079 19285 ts_tablet_manager.cc:1403] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.184s	user 0.016s	sys 0.016s
I20260709 12:14:13.596971 19286 consensus_queue.cc:260] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.616650 19286 ts_tablet_manager.cc:1434] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:14:13.618196 19286 tablet_bootstrap.cc:492] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:13.622030 19285 raft_consensus.cc:359] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.633996 19285 raft_consensus.cc:385] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.634366 19285 raft_consensus.cc:740] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.635052 19284 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:13.643621 19284 tablet_bootstrap.cc:654] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.649097 19285 consensus_queue.cc:260] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.654842 19285 ts_tablet_manager.cc:1434] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.003s
I20260709 12:14:13.655594 19283 raft_consensus.cc:359] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.656435 19283 raft_consensus.cc:385] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.656740 19283 raft_consensus.cc:740] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.657573 19283 consensus_queue.cc:260] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.663764 19285 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:13.660104 19283 ts_tablet_manager.cc:1434] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.002s
I20260709 12:14:13.676889 19283 tablet_bootstrap.cc:492] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:13.657044 19286 tablet_bootstrap.cc:654] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.687422 19522 raft_consensus.cc:493] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.688284 19522 raft_consensus.cc:515] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.694136 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718f20f68825449c836f67bd5a4bae55" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:13.696488 19055 leader_election.cc:343] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.697813 19522 leader_election.cc:290] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:13.718655 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718f20f68825449c836f67bd5a4bae55" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:13.719650 19239 raft_consensus.cc:2468] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:13.720835 19056 leader_election.cc:304] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:13.722568 19621 raft_consensus.cc:2804] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:13.725409 19621 raft_consensus.cc:493] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.725772 19014 raft_consensus.cc:3060] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.727829 19618 raft_consensus.cc:493] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.728418 19618 raft_consensus.cc:515] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.729573 19621 raft_consensus.cc:3060] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.730794 19618 leader_election.cc:290] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:13.732878 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d133466fbe4df8a6e09156cec6acf4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:13.733376 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d133466fbe4df8a6e09156cec6acf4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:13.734169 19090 raft_consensus.cc:2468] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
W20260709 12:14:13.734616 19203 leader_election.cc:343] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.732923 19284 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:13.735978 19284 ts_tablet_manager.cc:1403] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.101s	user 0.014s	sys 0.009s
I20260709 12:14:13.735972 19204 leader_election.cc:304] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:13.738662 19368 raft_consensus.cc:2804] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:13.739122 19368 raft_consensus.cc:493] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.739490 19368 raft_consensus.cc:3060] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.739537 19284 raft_consensus.cc:359] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.740334 19284 raft_consensus.cc:385] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.740612 19284 raft_consensus.cc:740] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.754148 19621 raft_consensus.cc:515] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.758643 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718f20f68825449c836f67bd5a4bae55" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:13.759383 19239 raft_consensus.cc:3060] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.760244 19621 leader_election.cc:290] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:13.763599 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718f20f68825449c836f67bd5a4bae55" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:13.763661 19368 raft_consensus.cc:515] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.767158 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d133466fbe4df8a6e09156cec6acf4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:13.768777 19203 leader_election.cc:343] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.771490 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d133466fbe4df8a6e09156cec6acf4" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:13.772344 19089 raft_consensus.cc:3060] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.775019 19285 tablet_bootstrap.cc:654] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.776729 19638 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.777261 19638 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.779788 19283 tablet_bootstrap.cc:654] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.779995 19284 consensus_queue.cc:260] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.782879 19284 ts_tablet_manager.cc:1434] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 12:14:13.783140 19016 raft_consensus.cc:3060] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.784284 19284 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:13.787915 19089 raft_consensus.cc:2468] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:13.777226 19239 raft_consensus.cc:2468] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:13.789130 19368 leader_election.cc:290] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:13.792435 19204 leader_election.cc:304] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:13.792709 19056 leader_election.cc:304] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:13.793502 19620 raft_consensus.cc:2804] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:13.794134 19522 raft_consensus.cc:2804] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:13.794849 19522 raft_consensus.cc:697] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:13.796103 19522 consensus_queue.cc:237] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.797335 19016 raft_consensus.cc:2468] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:13.787714 19013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:13.799501 19235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:13.802946 19235 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
W20260709 12:14:13.803761 19129 leader_election.cc:343] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.782908 19638 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:13.804864 19130 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:13.807338 19638 raft_consensus.cc:2804] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:13.807763 19638 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.808127 19638 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.818440 19285 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:13.819340 19285 ts_tablet_manager.cc:1403] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.156s	user 0.019s	sys 0.000s
I20260709 12:14:13.820190 19286 tablet_bootstrap.cc:492] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:13.820942 19286 ts_tablet_manager.cc:1403] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.203s	user 0.012s	sys 0.012s
I20260709 12:14:13.824538 19284 tablet_bootstrap.cc:654] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.827958 19285 raft_consensus.cc:359] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.828845 19285 raft_consensus.cc:385] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.829157 19285 raft_consensus.cc:740] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.823699 19620 raft_consensus.cc:697] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:13.829887 19285 consensus_queue.cc:260] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.831166 19620 consensus_queue.cc:237] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:13.832947 19286 raft_consensus.cc:359] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.834371 19283 tablet_bootstrap.cc:492] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:13.834970 19283 ts_tablet_manager.cc:1403] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.158s	user 0.016s	sys 0.003s
I20260709 12:14:13.840111 19285 ts_tablet_manager.cc:1434] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:14:13.833753 19286 raft_consensus.cc:385] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.841529 19285 tablet_bootstrap.cc:492] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:13.841681 19286 raft_consensus.cc:740] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.842072 19283 raft_consensus.cc:359] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.842810 19283 raft_consensus.cc:385] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.842727 19286 consensus_queue.cc:260] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.843546 19283 raft_consensus.cc:740] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.844338 19283 consensus_queue.cc:260] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.844076 18908 catalog_manager.cc:5697] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:13.840044 19638 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.845475 19286 ts_tablet_manager.cc:1434] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 12:14:13.880057 19285 tablet_bootstrap.cc:654] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.880503 19286 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:13.852993 19283 ts_tablet_manager.cc:1434] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:14:13.882493 19284 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:13.883119 19284 ts_tablet_manager.cc:1403] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.014s
I20260709 12:14:13.890444 19283 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:13.896905 19284 raft_consensus.cc:359] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.897827 19284 raft_consensus.cc:385] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.898154 19284 raft_consensus.cc:740] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.898424 19286 tablet_bootstrap.cc:654] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.898905 19284 consensus_queue.cc:260] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:13.906847 19579 raft_consensus.cc:493] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.907847 19579 raft_consensus.cc:515] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.913789 19284 ts_tablet_manager.cc:1434] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 12:14:13.913516 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:13.914722 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:13.915666 19239 raft_consensus.cc:2468] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:13.917052 19284 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:13.917553 19130 leader_election.cc:304] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
W20260709 12:14:13.919006 19130 leader_election.cc:343] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:13.923256 19581 raft_consensus.cc:2804] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:13.923875 19581 raft_consensus.cc:493] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.924315 19581 raft_consensus.cc:3060] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.926484 19285 tablet_bootstrap.cc:492] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:13.927109 19285 ts_tablet_manager.cc:1403] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.086s	user 0.025s	sys 0.023s
I20260709 12:14:13.931380 19285 raft_consensus.cc:359] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.932157 19285 raft_consensus.cc:385] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.932459 19285 raft_consensus.cc:740] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.906311 18911 catalog_manager.cc:5697] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:13.936790 19239 raft_consensus.cc:3060] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:13.910771 19579 leader_election.cc:290] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:13.943027 19357 consensus_queue.cc:1048] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:13.969029 19284 tablet_bootstrap.cc:654] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:13.980069 19285 consensus_queue.cc:260] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:13.981282 19284 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:13.981910 19284 ts_tablet_manager.cc:1403] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.010s
I20260709 12:14:13.988803 19285 ts_tablet_manager.cc:1434] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.002s
I20260709 12:14:13.990118 19285 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:13.985190 19284 raft_consensus.cc:359] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:13.993726 19284 raft_consensus.cc:385] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:13.994056 19284 raft_consensus.cc:740] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:13.994891 19284 consensus_queue.cc:260] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.003144 19579 raft_consensus.cc:493] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.003801 19579 raft_consensus.cc:515] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.005904 19235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:14.007081 19235 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.018551 19284 ts_tablet_manager.cc:1434] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:14:14.020301 19524 raft_consensus.cc:493] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.020960 19524 raft_consensus.cc:515] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.032976 19284 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:14.034489 19638 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:14.040491 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:14.040993 19284 tablet_bootstrap.cc:654] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.041538 19014 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.042187 19620 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.042421 19235 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:14.042773 19620 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.045130 19285 tablet_bootstrap.cc:654] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.048966 19620 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:14.049155 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:14.049875 19090 raft_consensus.cc:2468] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:14.050964 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:14.053583 19638 consensus_queue.cc:1048] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:14:14.057569 19283 tablet_bootstrap.cc:654] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.060022 19130 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:14.053658 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:14.054054 19204 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:14.061582 19604 raft_consensus.cc:2804] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:14.063112 19620 raft_consensus.cc:2804] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:14.063617 19620 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.040442 19581 raft_consensus.cc:515] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.064056 19620 raft_consensus.cc:3060] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:14.071000 19604 raft_consensus.cc:697] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:14.084070 19604 consensus_queue.cc:237] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.085863 19285 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:14.086479 19285 ts_tablet_manager.cc:1403] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.097s	user 0.010s	sys 0.005s
I20260709 12:14:14.088100 19524 leader_election.cc:290] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:14.095562 19285 raft_consensus.cc:359] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.096773 19285 raft_consensus.cc:385] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.097184 19285 raft_consensus.cc:740] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.104791 19162 raft_consensus.cc:2393] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
I20260709 12:14:14.106662 19579 leader_election.cc:290] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:14.108438 19581 leader_election.cc:290] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:14.074203 19638 consensus_queue.cc:1048] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:14:14.111949 19596 consensus_queue.cc:1048] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:14.123226 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:14.123929 19239 raft_consensus.cc:2468] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
W20260709 12:14:14.127061 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 66024.7us
I20260709 12:14:14.135834 19285 consensus_queue.cc:260] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.138062 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:14.138732 19239 raft_consensus.cc:3060] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:14.141033 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:14.144024 19056 leader_election.cc:304] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:14.147415 19620 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.148727 19235 raft_consensus.cc:1240] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Rejecting Update request from peer 6cab1cfd695145f992765d39dbe662c7 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:14:14.150813 19285 ts_tablet_manager.cc:1434] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.064s	user 0.010s	sys 0.002s
I20260709 12:14:14.163204 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:14.163782 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5f4455e6964b99b522e009ca88b6d8" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:14.164779 19089 raft_consensus.cc:2468] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:14.154233 19606 raft_consensus.cc:2804] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:14.165485 19606 raft_consensus.cc:493] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.165856 19606 raft_consensus.cc:3060] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.166409 19285 tablet_bootstrap.cc:492] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:14.174249 19285 tablet_bootstrap.cc:654] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.174353 19130 leader_election.cc:304] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:14.176327 19620 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:14.184121 19658 raft_consensus.cc:2804] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:14.184645 19658 raft_consensus.cc:493] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.184861 19014 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:14.188902 19239 raft_consensus.cc:2468] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:14.185104 19658 raft_consensus.cc:3060] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.186033 19579 consensus_queue.cc:1059] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:14.186407 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:14.187758 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:14.191691 19130 leader_election.cc:304] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:14.193373 19579 raft_consensus.cc:3055] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:14.198701 19579 raft_consensus.cc:740] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:14.206314 19579 consensus_queue.cc:260] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.207782 19579 raft_consensus.cc:3060] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:14.211998 19286 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:14.212743 19286 ts_tablet_manager.cc:1403] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.332s	user 0.053s	sys 0.055s
I20260709 12:14:14.217800 19204 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:14.198927 19654 raft_consensus.cc:2804] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:14.219305 19620 raft_consensus.cc:2749] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:14:14.219764 19654 raft_consensus.cc:697] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:14.220809 19654 consensus_queue.cc:237] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.211165 18911 catalog_manager.cc:5697] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:14.216746 19286 raft_consensus.cc:359] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.227000 19286 raft_consensus.cc:385] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.227478 19286 raft_consensus.cc:740] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
W20260709 12:14:14.197595 19129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 130470us
I20260709 12:14:14.233390 19284 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:14.234413 19284 ts_tablet_manager.cc:1403] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.202s	user 0.024s	sys 0.025s
I20260709 12:14:14.230455 19286 consensus_queue.cc:260] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.237629 19286 ts_tablet_manager.cc:1434] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
I20260709 12:14:14.239078 19286 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:14.240643 19606 raft_consensus.cc:515] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.246011 19573 raft_consensus.cc:493] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.260226 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5f4455e6964b99b522e009ca88b6d8" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:14.264621 19573 raft_consensus.cc:515] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.265666 19090 raft_consensus.cc:2393] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:14.266855 19606 leader_election.cc:290] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:14.284669 19286 tablet_bootstrap.cc:654] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.284730 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:14.286152 19239 raft_consensus.cc:2393] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 1: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
I20260709 12:14:14.281490 19573 leader_election.cc:290] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:14.292392 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
W20260709 12:14:14.294093 19129 leader_election.cc:343] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:14.294653 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54df033d2cbe422390e0fa4bf287ee14" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:14.297436 19284 raft_consensus.cc:359] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.299212 18980 leader_election.cc:304] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:14.298254 19284 raft_consensus.cc:385] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.300112 19284 raft_consensus.cc:740] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.300880 19284 consensus_queue.cc:260] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.301399 19573 raft_consensus.cc:2749] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:14.290832 19658 raft_consensus.cc:515] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.312736 19283 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:14.313465 19283 ts_tablet_manager.cc:1403] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.424s	user 0.019s	sys 0.013s
I20260709 12:14:14.315778 19285 tablet_bootstrap.cc:492] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:14.316365 19285 ts_tablet_manager.cc:1403] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.151s	user 0.013s	sys 0.015s
I20260709 12:14:14.317844 19284 ts_tablet_manager.cc:1434] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.083s	user 0.004s	sys 0.004s
I20260709 12:14:14.319132 19284 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:14.337284 19014 raft_consensus.cc:3060] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.344353 19284 tablet_bootstrap.cc:654] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.343624 18910 catalog_manager.cc:5697] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:14.347095 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5f4455e6964b99b522e009ca88b6d8" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:14.345865 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eff579fc38a94049a77efcb07b6bf35c" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:14.344308 19283 raft_consensus.cc:359] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.349368 19283 raft_consensus.cc:385] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.349495 19055 leader_election.cc:304] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:14.356072 19562 raft_consensus.cc:2749] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:14.349718 19283 raft_consensus.cc:740] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.357003 19283 consensus_queue.cc:260] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.362716 19283 ts_tablet_manager.cc:1434] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.003s
I20260709 12:14:14.363138 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5f4455e6964b99b522e009ca88b6d8" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:14.363377 19285 raft_consensus.cc:359] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:14.359097 19129 leader_election.cc:343] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:14.364472 19285 raft_consensus.cc:385] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.367267 19285 raft_consensus.cc:740] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.367683 19283 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:14.368136 19013 raft_consensus.cc:3060] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.363986 19090 raft_consensus.cc:3060] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.368150 19285 consensus_queue.cc:260] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.360088 19658 leader_election.cc:290] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:14.390162 19285 ts_tablet_manager.cc:1434] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.003s
I20260709 12:14:14.383834 19645 consensus_queue.cc:1048] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:14:14.398353 19524 consensus_queue.cc:1048] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:14.404768 19090 raft_consensus.cc:2468] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:14.411463 19284 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:14.414443 19284 ts_tablet_manager.cc:1403] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.096s	user 0.026s	sys 0.016s
I20260709 12:14:14.420791 19285 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:14.425725 19130 leader_election.cc:304] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:14.431629 19658 raft_consensus.cc:2804] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:14.432190 19658 raft_consensus.cc:697] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:14.433221 19658 consensus_queue.cc:237] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.436925 19620 consensus_queue.cc:1048] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:14.441231 19286 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:14.441907 19286 ts_tablet_manager.cc:1403] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.203s	user 0.010s	sys 0.010s
I20260709 12:14:14.459575 19283 tablet_bootstrap.cc:654] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.530139 19286 raft_consensus.cc:359] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.531623 19286 raft_consensus.cc:385] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.532110 19286 raft_consensus.cc:740] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.575953 19284 raft_consensus.cc:359] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.577111 19284 raft_consensus.cc:385] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.577507 19284 raft_consensus.cc:740] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.580335 19285 tablet_bootstrap.cc:654] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.578471 19284 consensus_queue.cc:260] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.588158 19286 consensus_queue.cc:260] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.604223 19606 raft_consensus.cc:493] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.609225 19606 raft_consensus.cc:515] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.610337 19283 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:14.610948 19283 ts_tablet_manager.cc:1403] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.244s	user 0.027s	sys 0.018s
I20260709 12:14:14.611590 19606 leader_election.cc:290] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:14.614719 19283 raft_consensus.cc:359] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.623709 19283 raft_consensus.cc:385] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.624037 19283 raft_consensus.cc:740] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.624840 19283 consensus_queue.cc:260] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.656636 19283 ts_tablet_manager.cc:1434] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 12:14:14.658874 19284 ts_tablet_manager.cc:1434] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.244s	user 0.009s	sys 0.001s
W20260709 12:14:14.659653 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 118226us
I20260709 12:14:14.666757 19286 ts_tablet_manager.cc:1434] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.224s	user 0.018s	sys 0.034s
I20260709 12:14:14.669044 19286 tablet_bootstrap.cc:492] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
W20260709 12:14:14.672868 19669 log.cc:927] Time spent T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:14:14.678323 19286 tablet_bootstrap.cc:654] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.684666 19283 tablet_bootstrap.cc:492] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:14.689988 19285 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:14.690685 19285 ts_tablet_manager.cc:1403] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.270s	user 0.021s	sys 0.018s
I20260709 12:14:14.691782 19677 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.693238 19677 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.694278 19283 tablet_bootstrap.cc:654] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:14.695554 19677 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:14.695544 19285 raft_consensus.cc:359] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.696625 19285 raft_consensus.cc:385] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.697104 19285 raft_consensus.cc:740] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
W20260709 12:14:14.763862 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 71753.2us
I20260709 12:14:14.784130 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9e5cd30214f99a71d1f25f34dae84" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:14.790248 19284 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:14.786172 19162 raft_consensus.cc:2468] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:14.796257 19524 raft_consensus.cc:493] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.797447 19524 raft_consensus.cc:515] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.807097 19596 raft_consensus.cc:493] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.808350 19596 raft_consensus.cc:515] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:14.825845 19522 consensus_queue.cc:1048] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:14.835570 19286 tablet_bootstrap.cc:492] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:14.850337 19283 tablet_bootstrap.cc:492] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:14.852452 19283 ts_tablet_manager.cc:1403] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.000s
I20260709 12:14:14.855497 19579 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.856609 19579 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:14.866851 19283 raft_consensus.cc:359] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.869668 19596 leader_election.cc:290] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:14.871415 19055 leader_election.cc:304] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:14.869820 19283 raft_consensus.cc:385] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:14.873832 19606 raft_consensus.cc:2804] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:14.874715 19606 raft_consensus.cc:493] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.873896 19283 raft_consensus.cc:740] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:14.879491 19285 consensus_queue.cc:260] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.879560 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5f4455e6964b99b522e009ca88b6d8" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:14.844164 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:14.883919 19162 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:14.885751 19283 consensus_queue.cc:260] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:14.887084 19286 ts_tablet_manager.cc:1403] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.219s	user 0.017s	sys 0.001s
I20260709 12:14:14.904609 19579 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:14.858213 18911 catalog_manager.cc:5697] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:14.923544 19658 consensus_queue.cc:1048] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:14.875309 19606 raft_consensus.cc:3060] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:14.928133 19283 ts_tablet_manager.cc:1434] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.003s
I20260709 12:14:14.930423 19283 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:14.938504 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9e5cd30214f99a71d1f25f34dae84" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:14.939815 19284 tablet_bootstrap.cc:700] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Time spent opening tablet: real 0.117s	user 0.004s	sys 0.000s
I20260709 12:14:14.941020 19284 tablet_bootstrap.cc:654] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:14.941053 19129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 75665.4us
I20260709 12:14:14.956485 19672 raft_consensus.cc:493] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.957123 19672 raft_consensus.cc:515] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:14.982503 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:14.980683 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:14.988662 19524 leader_election.cc:290] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:14.995813 19672 leader_election.cc:290] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:14.996609 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 177797us
I20260709 12:14:15.000699 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b5f4455e6964b99b522e009ca88b6d8" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:15.001926 19088 raft_consensus.cc:2393] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
W20260709 12:14:15.005901 19203 leader_election.cc:343] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.011363 19283 tablet_bootstrap.cc:654] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.013605 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de7d5ebe62f4f1198a4076ef2c81d34" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:14.981170 19285 ts_tablet_manager.cc:1434] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.290s	user 0.006s	sys 0.004s
I20260709 12:14:15.017495 19285 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:15.026875 19604 raft_consensus.cc:493] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.030149 19604 raft_consensus.cc:515] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.033187 19604 leader_election.cc:290] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.059155 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.060250 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de7d5ebe62f4f1198a4076ef2c81d34" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:15.061079 19090 raft_consensus.cc:2468] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:15.061172 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:15.062172 19163 raft_consensus.cc:2468] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:15.063025 19204 leader_election.cc:304] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
W20260709 12:14:15.066129 19055 leader_election.cc:343] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.066138 19672 raft_consensus.cc:2804] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:15.068153 19672 raft_consensus.cc:493] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.068535 19672 raft_consensus.cc:3060] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.078308 19284 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:15.079007 19284 ts_tablet_manager.cc:1403] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.290s	user 0.028s	sys 0.034s
W20260709 12:14:15.046141 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 48489.9us
I20260709 12:14:15.082932 19284 raft_consensus.cc:359] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.084290 19284 raft_consensus.cc:385] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.084838 19284 raft_consensus.cc:740] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.010351 19606 raft_consensus.cc:515] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:15.086369 19284 consensus_queue.cc:260] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.088438 19203 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:15.089751 19677 raft_consensus.cc:2804] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:15.090399 19677 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.089787 19284 ts_tablet_manager.cc:1434] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:14:15.090353 19606 leader_election.cc:290] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:15.111413 19283 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:15.114368 19283 ts_tablet_manager.cc:1403] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.185s	user 0.017s	sys 0.021s
I20260709 12:14:15.115957 19284 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:15.091696 19677 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:15.120635 19130 leader_election.cc:343] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.119511 19604 consensus_queue.cc:1048] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:14:15.127156 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.116456 19579 consensus_queue.cc:1048] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:14:15.132351 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9e5cd30214f99a71d1f25f34dae84" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:15.133180 19162 raft_consensus.cc:3060] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.136595 18980 leader_election.cc:304] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:15.137485 19055 leader_election.cc:304] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:15.138752 19573 raft_consensus.cc:3060] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.140554 19686 raft_consensus.cc:2804] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:15.141036 19686 raft_consensus.cc:493] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.141407 19686 raft_consensus.cc:3060] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:15.143421 19129 leader_election.cc:343] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.144093 19129 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:15.146905 19286 raft_consensus.cc:359] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.147758 19286 raft_consensus.cc:385] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.148061 19286 raft_consensus.cc:740] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.149166 19286 consensus_queue.cc:260] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.151223 19579 raft_consensus.cc:2749] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:15.157696 19286 ts_tablet_manager.cc:1434] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.270s	user 0.010s	sys 0.004s
W20260709 12:14:15.159489 19203 leader_election.cc:343] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.153214 19283 raft_consensus.cc:359] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.163652 19283 raft_consensus.cc:385] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.164130 19283 raft_consensus.cc:740] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.167546 19283 consensus_queue.cc:260] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.168598 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.180907 19238 raft_consensus.cc:2468] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:15.175540 19016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9e5cd30214f99a71d1f25f34dae84" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:15.187736 19285 tablet_bootstrap.cc:654] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:15.188000 19055 leader_election.cc:343] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.196117 19162 raft_consensus.cc:2468] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:15.197592 19055 leader_election.cc:304] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:15.198571 19606 raft_consensus.cc:2804] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:15.202224 19286 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:15.202150 19606 raft_consensus.cc:697] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:15.205060 19606 consensus_queue.cc:237] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:15.213833 19677 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:15.216063 19130 leader_election.cc:343] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.220072 19284 tablet_bootstrap.cc:654] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.216094 19283 ts_tablet_manager.cc:1434] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.000s
I20260709 12:14:15.241012 19573 raft_consensus.cc:2749] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:15.249020 19283 tablet_bootstrap.cc:492] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:15.250677 19656 consensus_queue.cc:1048] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:15.254374 19286 tablet_bootstrap.cc:654] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.255863 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:15.256346 19677 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:15.259999 19656 consensus_queue.cc:1048] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:15.266597 19656 raft_consensus.cc:493] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.267184 19672 raft_consensus.cc:515] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.267208 19656 raft_consensus.cc:515] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.279278 19604 consensus_queue.cc:1048] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:14:15.296005 19656 leader_election.cc:290] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:15.304589 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de7d5ebe62f4f1198a4076ef2c81d34" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:15.305514 19089 raft_consensus.cc:3060] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.313218 19686 raft_consensus.cc:515] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.322404 19285 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:15.326375 19285 ts_tablet_manager.cc:1403] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.309s	user 0.017s	sys 0.005s
I20260709 12:14:15.307322 19672 leader_election.cc:290] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:15.354257 19088 raft_consensus.cc:2393] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 1: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:15.354560 19162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:15.355412 19162 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.356600 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de7d5ebe62f4f1198a4076ef2c81d34" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:15.357542 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.355422 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:15.365990 19130 leader_election.cc:343] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.366928 19015 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.371776 19130 leader_election.cc:304] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:15.372697 19579 raft_consensus.cc:3060] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.373667 19686 leader_election.cc:290] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.375293 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:15.376178 19237 raft_consensus.cc:3060] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.382687 19285 raft_consensus.cc:359] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.383745 19285 raft_consensus.cc:385] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.384243 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:15.384238 19285 raft_consensus.cc:740] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.411875 19009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:15.412241 19285 consensus_queue.cc:260] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:15.413793 19129 leader_election.cc:343] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.414460 19129 leader_election.cc:304] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:15.415591 19581 raft_consensus.cc:2749] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:14:15.419139 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 52904.4us
W20260709 12:14:15.419883 19203 leader_election.cc:343] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.421505 19089 raft_consensus.cc:2468] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:15.423601 19204 leader_election.cc:304] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:15.425313 19285 ts_tablet_manager.cc:1434] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.002s
I20260709 12:14:15.425447 19283 tablet_bootstrap.cc:654] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.426699 19285 tablet_bootstrap.cc:492] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:15.431074 19645 raft_consensus.cc:2804] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:15.431957 19645 raft_consensus.cc:697] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:15.433004 19645 consensus_queue.cc:237] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.469612 19285 tablet_bootstrap.cc:654] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.472957 19162 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:15.474330 19284 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:15.475072 19284 ts_tablet_manager.cc:1403] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.360s	user 0.016s	sys 0.006s
I20260709 12:14:15.476323 19686 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.476953 19686 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.488740 19203 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:15.496894 19686 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:15.497006 19672 raft_consensus.cc:2804] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:15.502382 19672 raft_consensus.cc:697] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:15.503463 19672 consensus_queue.cc:237] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.509773 19015 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:15.513144 19579 raft_consensus.cc:2749] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:15.513526 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:15.522058 19056 leader_election.cc:343] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.525182 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:15.526829 19055 leader_election.cc:343] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.527319 19055 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:15.522271 19286 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:15.529458 19286 ts_tablet_manager.cc:1403] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.350s	user 0.026s	sys 0.037s
W20260709 12:14:15.542531 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 46707us
I20260709 12:14:15.550705 19163 raft_consensus.cc:2468] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:15.528424 19606 raft_consensus.cc:2749] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:15.571919 19237 raft_consensus.cc:2468] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
W20260709 12:14:15.574167 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:15.584371 19285 tablet_bootstrap.cc:492] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:15.583981 19286 raft_consensus.cc:359] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.584991 19286 raft_consensus.cc:385] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.585361 19286 raft_consensus.cc:740] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.586292 19286 consensus_queue.cc:260] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.585677 19056 leader_election.cc:304] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:15.590806 19286 ts_tablet_manager.cc:1434] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 12:14:15.591629 19686 raft_consensus.cc:2804] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:15.592211 19686 raft_consensus.cc:697] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:15.593196 19686 consensus_queue.cc:237] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.595464 19284 raft_consensus.cc:359] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.596221 19284 raft_consensus.cc:385] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.596530 19284 raft_consensus.cc:740] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.597335 19284 consensus_queue.cc:260] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.599777 19284 ts_tablet_manager.cc:1434] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.124s	user 0.006s	sys 0.001s
I20260709 12:14:15.601001 19284 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:15.597307 19286 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:15.616175 19285 ts_tablet_manager.cc:1403] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.190s	user 0.024s	sys 0.016s
I20260709 12:14:15.620270 19285 raft_consensus.cc:359] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.622258 19285 raft_consensus.cc:385] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.622611 19285 raft_consensus.cc:740] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.631608 19285 consensus_queue.cc:260] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.643265 19285 ts_tablet_manager.cc:1434] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 12:14:15.647957 19285 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:15.645838 19283 tablet_bootstrap.cc:492] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:15.650558 19283 ts_tablet_manager.cc:1403] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.402s	user 0.035s	sys 0.049s
I20260709 12:14:15.653218 18910 catalog_manager.cc:5697] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:15.656976 19283 raft_consensus.cc:359] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:15.658550 19055 consensus_peers.cc:597] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:15.667929 19283 raft_consensus.cc:385] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.668521 19283 raft_consensus.cc:740] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.669718 19284 tablet_bootstrap.cc:654] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.669802 19285 tablet_bootstrap.cc:654] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.676121 19678 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:15.676858 19678 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.682552 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:15.682667 19678 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.682873 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.684641 19164 raft_consensus.cc:2468] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:15.684981 19239 raft_consensus.cc:2393] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 2: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:15.686848 19055 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:15.687780 19686 raft_consensus.cc:2804] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:15.688164 19686 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:15.688531 19686 raft_consensus.cc:3060] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:15.694468 19286 tablet_bootstrap.cc:654] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.705113 19284 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:15.707077 19285 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:15.708729 19285 ts_tablet_manager.cc:1403] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.009s
I20260709 12:14:15.712160 19285 raft_consensus.cc:359] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:15.712947 19285 raft_consensus.cc:385] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.713294 19285 raft_consensus.cc:740] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.714120 19285 consensus_queue.cc:260] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:15.699193 19283 consensus_queue.cc:260] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.718142 19285 ts_tablet_manager.cc:1434] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:14:15.719502 19285 tablet_bootstrap.cc:492] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:15.720631 19678 consensus_queue.cc:1048] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:15.721421 19283 ts_tablet_manager.cc:1434] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.002s
I20260709 12:14:15.722604 19283 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:15.714583 19686 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.725312 19686 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.715559 19284 ts_tablet_manager.cc:1403] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.115s	user 0.018s	sys 0.008s
I20260709 12:14:15.726352 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:15.726418 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:15.727851 19239 raft_consensus.cc:2393] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 666fef4f7e5a4c32813dad8dfade8561 in current term 2: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:15.729151 19284 raft_consensus.cc:359] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.730974 19284 raft_consensus.cc:385] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.731357 19284 raft_consensus.cc:740] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.733570 19284 consensus_queue.cc:260] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.744846 19163 raft_consensus.cc:2468] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:15.746533 19055 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:15.753158 19678 raft_consensus.cc:2804] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:15.754460 19678 raft_consensus.cc:697] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:15.757859 19286 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:15.758472 19286 ts_tablet_manager.cc:1403] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.006s
I20260709 12:14:15.761701 19286 raft_consensus.cc:359] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.762411 19286 raft_consensus.cc:385] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.764073 19686 raft_consensus.cc:493] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.766927 19286 raft_consensus.cc:740] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.767436 19283 tablet_bootstrap.cc:654] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.771781 19284 ts_tablet_manager.cc:1434] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 12:14:15.773075 19284 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:15.775899 19686 raft_consensus.cc:515] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.782416 19686 leader_election.cc:290] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.776885 19678 consensus_queue.cc:237] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.783079 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67045c0c43a64d4a896e1928eda8289f" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:15.783665 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67045c0c43a64d4a896e1928eda8289f" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.783875 19160 raft_consensus.cc:2468] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
W20260709 12:14:15.785514 19056 leader_election.cc:343] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.786373 19055 leader_election.cc:304] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:15.779309 19286 consensus_queue.cc:260] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.779887 19596 raft_consensus.cc:493] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.791069 19524 raft_consensus.cc:2804] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:15.791599 19524 raft_consensus.cc:493] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.791210 19596 raft_consensus.cc:515] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.792006 19524 raft_consensus.cc:3060] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.792740 19286 ts_tablet_manager.cc:1434] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.004s
I20260709 12:14:15.793973 19596 leader_election.cc:290] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:15.794030 19286 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:15.799134 19285 tablet_bootstrap.cc:654] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.800115 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de7d5ebe62f4f1198a4076ef2c81d34" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:15.801079 19091 raft_consensus.cc:2393] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 1: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:15.802076 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de7d5ebe62f4f1198a4076ef2c81d34" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.804311 18980 leader_election.cc:304] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:15.807646 19393 raft_consensus.cc:3060] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.837451 19524 raft_consensus.cc:515] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:15.835045 18911 catalog_manager.cc:5697] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:15.848224 19393 raft_consensus.cc:2749] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:15.880503 19524 leader_election.cc:290] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.881299 19286 tablet_bootstrap.cc:654] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.882386 19284 tablet_bootstrap.cc:654] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:15.895339 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67045c0c43a64d4a896e1928eda8289f" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:15.896157 19163 raft_consensus.cc:3060] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.901427 19672 consensus_queue.cc:1048] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:15.908291 18911 catalog_manager.cc:5697] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:15.909709 19393 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.916366 19672 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.930078 19283 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:15.935695 19283 ts_tablet_manager.cc:1403] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.213s	user 0.029s	sys 0.015s
I20260709 12:14:15.935353 19393 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.935484 19672 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:15.937692 19393 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.931440 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67045c0c43a64d4a896e1928eda8289f" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
W20260709 12:14:15.940207 19056 leader_election.cc:343] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.940208 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:15.940843 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:15.939227 19283 raft_consensus.cc:359] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.941614 19164 raft_consensus.cc:2468] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:15.941743 19091 raft_consensus.cc:2468] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:15.941929 19283 raft_consensus.cc:385] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.942288 19283 raft_consensus.cc:740] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:15.942965 18979 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:15.943306 19204 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:15.944006 19393 raft_consensus.cc:2804] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:15.944334 19707 raft_consensus.cc:2804] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:15.937996 19672 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:15.944444 19393 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.945927 19283 consensus_queue.cc:260] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.947796 19393 raft_consensus.cc:3060] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.945182 19163 raft_consensus.cc:2468] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:15.951171 19055 leader_election.cc:304] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:15.946254 19683 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.957082 19683 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.959448 19683 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:15.954571 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:15.962193 19009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:15.954483 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:15.953882 19678 raft_consensus.cc:2804] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:15.956429 19283 ts_tablet_manager.cc:1434] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:14:15.946544 19707 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.964811 19707 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.964968 19678 raft_consensus.cc:697] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:15.966060 19678 consensus_queue.cc:237] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:15.963847 19203 leader_election.cc:343] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:15.963200 19239 raft_consensus.cc:2468] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 0.
I20260709 12:14:15.968905 19283 tablet_bootstrap.cc:492] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:15.973416 19705 consensus_queue.cc:1048] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:15.973958 19393 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:15.976161 19393 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:15.977243 19009 raft_consensus.cc:2393] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 1: Already voted for candidate 3fcac1aec8df4c599b4c67550ce347e1 in this term.
I20260709 12:14:15.980549 19285 tablet_bootstrap.cc:492] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:15.981165 19285 ts_tablet_manager.cc:1403] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.262s	user 0.014s	sys 0.003s
I20260709 12:14:15.981473 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:15.981675 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:15.982240 19164 raft_consensus.cc:3060] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.982378 19239 raft_consensus.cc:3060] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:15.983547 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:15.991487 19203 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:15.991781 19284 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:15.992353 19284 ts_tablet_manager.cc:1403] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.220s	user 0.041s	sys 0.013s
I20260709 12:14:15.996673 19284 raft_consensus.cc:359] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:15.997504 19284 raft_consensus.cc:385] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:15.997814 19284 raft_consensus.cc:740] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.001096 19727 raft_consensus.cc:493] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.000926 19562 consensus_queue.cc:1048] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.001770 19727 raft_consensus.cc:515] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:16.003472 19524 consensus_queue.cc:1048] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:16.004156 19727 leader_election.cc:290] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:16.011606 19164 raft_consensus.cc:2468] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:16.013149 18979 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:16.017019 19286 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:16.019016 19286 ts_tablet_manager.cc:1403] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.225s	user 0.018s	sys 0.026s
I20260709 12:14:16.016746 18910 catalog_manager.cc:5697] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.014101 19393 raft_consensus.cc:2804] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:16.017134 19284 consensus_queue.cc:260] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:16.017373 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.020568 18910 catalog_manager.cc:5697] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 1 to 2, leader changed from 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.025990 19009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:16.031828 19645 consensus_queue.cc:1048] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:16.038131 19283 tablet_bootstrap.cc:654] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.042021 19285 raft_consensus.cc:359] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.043080 19285 raft_consensus.cc:385] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.043453 19285 raft_consensus.cc:740] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.049202 19239 raft_consensus.cc:2468] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 1.
I20260709 12:14:16.049952 19705 raft_consensus.cc:2749] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:16.059286 19285 consensus_queue.cc:260] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:16.060098 19055 leader_election.cc:343] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:16.060998 19055 leader_election.cc:343] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.061399 19055 leader_election.cc:304] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:16.062155 19285 ts_tablet_manager.cc:1434] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.081s	user 0.003s	sys 0.007s
I20260709 12:14:16.063463 19285 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:16.063225 19621 raft_consensus.cc:2749] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:16.075172 19393 raft_consensus.cc:697] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:16.077239 19393 consensus_queue.cc:237] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.081693 19286 raft_consensus.cc:359] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.082489 19286 raft_consensus.cc:385] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.082751 19286 raft_consensus.cc:740] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.085844 19707 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.087548 19286 consensus_queue.cc:260] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.124593 19286 ts_tablet_manager.cc:1434] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.105s	user 0.009s	sys 0.000s
I20260709 12:14:16.129453 19283 tablet_bootstrap.cc:492] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:16.130054 19283 ts_tablet_manager.cc:1403] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.161s	user 0.012s	sys 0.003s
I20260709 12:14:16.123723 19707 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:16.123373 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:16.139782 19286 tablet_bootstrap.cc:492] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:16.132421 19683 consensus_queue.cc:1048] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 12:14:16.151194 19203 leader_election.cc:343] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.155388 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:16.156211 19087 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.163403 19579 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.169292 19579 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.171677 19683 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.172286 19683 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.172670 19579 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:16.176668 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:16.179984 19130 leader_election.cc:343] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.183887 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.184988 19164 raft_consensus.cc:2410] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:14:16.188846 19284 ts_tablet_manager.cc:1434] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.196s	user 0.008s	sys 0.003s
I20260709 12:14:16.192330 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:16.184103 19683 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:16.194145 19707 raft_consensus.cc:493] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.198858 19707 raft_consensus.cc:515] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:16.206689 19129 leader_election.cc:343] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.208029 19129 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:16.208618 19284 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:16.208911 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.209628 19164 raft_consensus.cc:2468] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:16.210851 19203 leader_election.cc:304] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:16.202497 19524 raft_consensus.cc:493] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.221309 19283 raft_consensus.cc:359] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.222687 19283 raft_consensus.cc:385] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.223064 19283 raft_consensus.cc:740] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.220757 19672 raft_consensus.cc:2804] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:16.233389 19672 raft_consensus.cc:493] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.233855 19672 raft_consensus.cc:3060] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.217468 19656 raft_consensus.cc:2749] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:16.252068 19595 raft_consensus.cc:493] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.252761 19595 raft_consensus.cc:515] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:16.256757 19595 leader_election.cc:290] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:16.258893 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9eff763afc14d8aa7bbd4505c26d0c3" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:16.246289 19283 consensus_queue.cc:260] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.226688 19524 raft_consensus.cc:515] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.273190 19524 leader_election.cc:290] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:16.226861 19707 leader_election.cc:290] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:16.263804 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:16.269140 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9e5cd30214f99a71d1f25f34dae84" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.278117 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8747727903d14849a4d18d8676bbf505" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.279068 19160 raft_consensus.cc:2468] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 0.
I20260709 12:14:16.280644 19055 leader_election.cc:304] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:16.281982 19621 raft_consensus.cc:2804] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:16.282804 19621 raft_consensus.cc:493] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.283429 19621 raft_consensus.cc:3060] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.270759 19285 tablet_bootstrap.cc:654] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.265859 19204 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
W20260709 12:14:16.284816 19203 leader_election.cc:343] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.282001 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8747727903d14849a4d18d8676bbf505" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:16.291500 19284 tablet_bootstrap.cc:654] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.297410 19680 raft_consensus.cc:2749] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:16.303208 19672 raft_consensus.cc:515] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.306332 19672 leader_election.cc:290] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:16.311372 19056 leader_election.cc:343] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.311856 19286 tablet_bootstrap.cc:654] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.313087 19283 ts_tablet_manager.cc:1434] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.183s	user 0.009s	sys 0.001s
I20260709 12:14:16.316696 19087 raft_consensus.cc:2468] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:16.319079 19204 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:16.320075 19672 raft_consensus.cc:2804] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:16.320590 19672 raft_consensus.cc:697] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:16.321528 19672 consensus_queue.cc:237] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.336217 19009 raft_consensus.cc:3060] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.342381 19283 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:16.345816 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:16.353407 19163 raft_consensus.cc:1275] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:16.358656 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:16.359397 19164 raft_consensus.cc:3060] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:16.359665 19203 leader_election.cc:343] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.362691 19562 consensus_queue.cc:1048] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.371796 19164 raft_consensus.cc:2468] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:16.375473 19203 leader_election.cc:304] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:16.390054 19683 raft_consensus.cc:2804] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:16.406482 19283 tablet_bootstrap.cc:654] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.372699 19621 raft_consensus.cc:515] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.412516 18912 catalog_manager.cc:5697] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:16.413820 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8747727903d14849a4d18d8676bbf505" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:16.414589 19163 raft_consensus.cc:3060] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.399801 19683 raft_consensus.cc:697] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:16.421783 19683 consensus_queue.cc:237] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.425401 19286 tablet_bootstrap.cc:492] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:16.426074 19286 ts_tablet_manager.cc:1403] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.289s	user 0.012s	sys 0.011s
I20260709 12:14:16.432597 19163 raft_consensus.cc:2468] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:16.435329 19055 leader_election.cc:304] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:16.435747 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8747727903d14849a4d18d8676bbf505" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:16.435968 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9e5cd30214f99a71d1f25f34dae84" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:16.436517 19524 raft_consensus.cc:2804] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:16.437107 19524 raft_consensus.cc:697] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:16.438030 19524 consensus_queue.cc:237] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:16.448621 19056 leader_election.cc:343] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.453056 19286 raft_consensus.cc:359] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.453940 19286 raft_consensus.cc:385] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.454248 19286 raft_consensus.cc:740] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.455179 19286 consensus_queue.cc:260] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.430464 19621 leader_election.cc:290] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:16.462393 19238 raft_consensus.cc:1275] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:16.469205 19285 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:16.472191 19285 ts_tablet_manager.cc:1403] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.409s	user 0.017s	sys 0.009s
I20260709 12:14:16.473567 18980 leader_election.cc:304] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:16.475952 19284 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:16.476768 19284 ts_tablet_manager.cc:1403] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.268s	user 0.044s	sys 0.037s
I20260709 12:14:16.477771 19286 ts_tablet_manager.cc:1434] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.001s
I20260709 12:14:16.468864 19621 consensus_queue.cc:1048] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:14:16.501412 18910 catalog_manager.cc:5697] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.510134 19285 raft_consensus.cc:359] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.519825 19285 raft_consensus.cc:385] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.520238 19285 raft_consensus.cc:740] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.525091 19285 consensus_queue.cc:260] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.538837 19286 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:16.575459 19236 raft_consensus.cc:3060] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.576702 18911 catalog_manager.cc:5697] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.590924 19595 consensus_queue.cc:1048] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.592707 19621 consensus_queue.cc:1048] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.607671 19285 ts_tablet_manager.cc:1434] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.135s	user 0.008s	sys 0.001s
I20260709 12:14:16.625136 19284 raft_consensus.cc:359] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.628250 19284 raft_consensus.cc:385] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.628589 19284 raft_consensus.cc:740] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.630281 19284 consensus_queue.cc:260] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.633410 19284 ts_tablet_manager.cc:1434] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.156s	user 0.010s	sys 0.000s
I20260709 12:14:16.635093 19284 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:16.639858 19686 consensus_queue.cc:1048] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.641892 19393 raft_consensus.cc:2749] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:16.655259 19705 raft_consensus.cc:493] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.656603 19705 raft_consensus.cc:515] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:16.659128 19705 leader_election.cc:290] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:16.660398 19283 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:16.660969 19283 ts_tablet_manager.cc:1403] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.319s	user 0.027s	sys 0.010s
I20260709 12:14:16.662362 19285 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:16.664846 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67045c0c43a64d4a896e1928eda8289f" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:16.669783 19606 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.670311 19606 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.678241 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67045c0c43a64d4a896e1928eda8289f" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.682955 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:16.684645 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:16.685986 19055 leader_election.cc:343] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.687505 19286 tablet_bootstrap.cc:654] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:16.690876 19056 leader_election.cc:343] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.691146 19203 leader_election.cc:304] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:16.681492 19606 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:16.691460 19056 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:16.703171 19686 raft_consensus.cc:2749] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:16.710194 18910 catalog_manager.cc:5697] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.712342 19285 tablet_bootstrap.cc:654] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.710670 18908 catalog_manager.cc:5697] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.714485 19744 consensus_queue.cc:1048] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:16.712430 19283 raft_consensus.cc:359] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:16.719218 19283 raft_consensus.cc:385] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.723348 19283 raft_consensus.cc:740] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.733528 19284 tablet_bootstrap.cc:654] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.743932 19286 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:16.750622 19286 ts_tablet_manager.cc:1403] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.225s	user 0.008s	sys 0.014s
I20260709 12:14:16.738343 19283 consensus_queue.cc:260] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:16.755194 19286 raft_consensus.cc:359] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.756538 19286 raft_consensus.cc:385] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.756860 19286 raft_consensus.cc:740] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.757830 19286 consensus_queue.cc:260] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.760751 19286 ts_tablet_manager.cc:1434] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:14:16.761086 19283 ts_tablet_manager.cc:1434] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.100s	user 0.011s	sys 0.000s
I20260709 12:14:16.761988 19286 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:16.763193 19393 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.771267 19393 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.773413 19009 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.794672 19285 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:16.795727 19285 ts_tablet_manager.cc:1403] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.152s	user 0.013s	sys 0.002s
I20260709 12:14:16.796852 19672 raft_consensus.cc:2749] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:16.802187 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:16.798506 19705 consensus_queue.cc:1048] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:14:16.811241 19393 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:16.817009 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:16.826076 19705 consensus_queue.cc:1048] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.827446 19284 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:16.828146 19284 ts_tablet_manager.cc:1403] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.193s	user 0.014s	sys 0.034s
I20260709 12:14:16.828899 19286 tablet_bootstrap.cc:654] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.847481 18980 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:16.849493 19393 raft_consensus.cc:2749] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:16.848409 19285 raft_consensus.cc:359] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:16.851260 18960 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:16.851528 19285 raft_consensus.cc:385] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.851917 19285 raft_consensus.cc:740] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.852584 19283 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:16.852619 19285 consensus_queue.cc:260] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.870072 19645 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.879925 19645 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.884473 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:16.886394 19203 leader_election.cc:343] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.887868 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:16.888648 19163 raft_consensus.cc:2468] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:16.899422 19203 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
W20260709 12:14:16.904745 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:16.905198 19284 raft_consensus.cc:359] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.906112 19284 raft_consensus.cc:385] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.906432 19284 raft_consensus.cc:740] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.913525 19683 raft_consensus.cc:2804] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:16.919299 19683 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.919679 19683 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:16.923141 19285 ts_tablet_manager.cc:1434] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.127s	user 0.009s	sys 0.021s
I20260709 12:14:16.924437 19285 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: Bootstrap starting.
I20260709 12:14:16.907747 19284 consensus_queue.cc:260] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.882881 19645 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:16.932319 19055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 66272.3us
I20260709 12:14:16.936515 19656 raft_consensus.cc:493] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.937062 19656 raft_consensus.cc:515] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:16.939520 19656 leader_election.cc:290] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:16.946064 19770 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.946023 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:16.947770 19088 raft_consensus.cc:2468] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:16.948949 19286 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:16.946144 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:16.949508 19286 ts_tablet_manager.cc:1403] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.188s	user 0.014s	sys 0.002s
I20260709 12:14:16.949460 19130 leader_election.cc:304] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:16.950546 19656 raft_consensus.cc:2804] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:16.950949 19656 raft_consensus.cc:493] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.951319 19656 raft_consensus.cc:3060] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:16.952665 19129 leader_election.cc:343] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:16.953027 19286 raft_consensus.cc:359] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.953734 19286 raft_consensus.cc:385] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:16.954016 19286 raft_consensus.cc:740] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:16.955770 19286 consensus_queue.cc:260] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:16.957585 19768 log.cc:927] Time spent T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: Append to log took a long time: real 0.078s	user 0.003s	sys 0.000s
I20260709 12:14:16.958559 19286 ts_tablet_manager.cc:1434] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 12:14:16.947556 19770 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:16.959909 19286 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:16.959146 19683 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:16.992436 19283 tablet_bootstrap.cc:654] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.001428 19285 tablet_bootstrap.cc:654] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:16.997820 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:17.002131 19163 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.002759 19680 consensus_queue.cc:1048] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:16.992554 19683 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:16.999782 19770 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:17.001241 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:16.998569 19284 ts_tablet_manager.cc:1434] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.170s	user 0.007s	sys 0.002s
W20260709 12:14:17.009338 19056 leader_election.cc:343] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.012321 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:17.014250 19203 leader_election.cc:343] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.008858 19284 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:17.033174 19656 raft_consensus.cc:515] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.048518 19009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:17.050365 19163 raft_consensus.cc:2468] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:17.053388 19203 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:17.055017 19683 raft_consensus.cc:2804] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:17.055709 19683 raft_consensus.cc:697] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:17.056649 19683 consensus_queue.cc:237] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.075642 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:17.080945 19129 leader_election.cc:343] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.085629 19284 tablet_bootstrap.cc:654] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.093410 19680 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.094007 19680 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.097456 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:17.096302 19680 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.098237 19088 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:17.101691 19283 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:17.102301 19283 ts_tablet_manager.cc:1403] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.285s	user 0.018s	sys 0.000s
W20260709 12:14:17.080513 19055 leader_election.cc:343] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.116096 19055 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:17.116163 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:17.128751 19286 tablet_bootstrap.cc:654] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.126959 19774 raft_consensus.cc:2749] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:17.133864 19771 consensus_queue.cc:1048] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:17.127241 19204 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
W20260709 12:14:17.135926 19203 leader_election.cc:343] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.137488 19680 raft_consensus.cc:2804] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:17.138293 19680 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.138697 19680 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:17.142285 19056 consensus_peers.cc:597] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373): Couldn't send request to peer 2c0b3983ae6c48108f6f3632116cf5a8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:17.143180 19656 leader_election.cc:290] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.158255 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:17.159245 19087 raft_consensus.cc:3060] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.163625 18911 catalog_manager.cc:5697] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:17.172345 19087 raft_consensus.cc:2468] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:17.180517 19130 leader_election.cc:304] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:17.182053 19285 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: No bootstrap required, opened a new log
I20260709 12:14:17.182754 19285 ts_tablet_manager.cc:1403] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: Time spent bootstrapping tablet: real 0.259s	user 0.016s	sys 0.003s
I20260709 12:14:17.183936 19579 raft_consensus.cc:2804] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:17.177242 19283 raft_consensus.cc:359] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.188066 19283 raft_consensus.cc:385] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.188396 19283 raft_consensus.cc:740] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.189354 19283 consensus_queue.cc:260] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.192586 19283 ts_tablet_manager.cc:1434] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.090s	user 0.006s	sys 0.004s
I20260709 12:14:17.195927 19393 raft_consensus.cc:493] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.196624 19393 raft_consensus.cc:515] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.199492 19283 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7: Bootstrap starting.
I20260709 12:14:17.200968 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:17.203958 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:17.206121 18979 leader_election.cc:304] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:17.207027 19744 raft_consensus.cc:2749] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:17.211032 19579 raft_consensus.cc:697] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:17.212273 19393 leader_election.cc:290] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:17.212256 19579 consensus_queue.cc:237] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.215873 19285 raft_consensus.cc:359] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.220252 19285 raft_consensus.cc:385] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.228350 19285 raft_consensus.cc:740] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.223454 19283 tablet_bootstrap.cc:654] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.230427 19285 consensus_queue.cc:260] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.223031 19286 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:17.234638 19286 ts_tablet_manager.cc:1403] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.275s	user 0.029s	sys 0.013s
I20260709 12:14:17.236641 19284 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:17.237421 19284 ts_tablet_manager.cc:1403] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.229s	user 0.021s	sys 0.022s
I20260709 12:14:17.242592 19284 raft_consensus.cc:359] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.243395 19284 raft_consensus.cc:385] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.243685 19284 raft_consensus.cc:740] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.244767 19284 consensus_queue.cc:260] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.249734 19284 ts_tablet_manager.cc:1434] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:14:17.251492 19284 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:17.260708 19285 ts_tablet_manager.cc:1434] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 12:14:17.233116 19680 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.264529 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:17.265242 19015 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.268699 19579 raft_consensus.cc:493] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.269341 19579 raft_consensus.cc:515] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.271731 19579 leader_election.cc:290] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.274329 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:17.275913 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:17.276736 19088 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 0.
I20260709 12:14:17.277539 18912 catalog_manager.cc:5697] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.259476 19286 raft_consensus.cc:359] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.278606 19130 leader_election.cc:304] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:17.279206 19286 raft_consensus.cc:385] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.279577 19286 raft_consensus.cc:740] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.280815 19015 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:17.281721 19680 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.282236 19203 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
W20260709 12:14:17.283149 19129 leader_election.cc:343] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.283035 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:17.283946 19088 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.285815 19286 consensus_queue.cc:260] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.287017 19683 raft_consensus.cc:2804] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:17.287562 19683 raft_consensus.cc:697] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:17.288528 19683 consensus_queue.cc:237] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.290141 19284 tablet_bootstrap.cc:654] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.291393 19782 raft_consensus.cc:2804] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:17.291899 19782 raft_consensus.cc:493] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.292248 19782 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.295774 19286 ts_tablet_manager.cc:1434] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 12:14:17.297075 19286 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:17.298950 19284 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:17.299540 19284 ts_tablet_manager.cc:1403] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.004s
I20260709 12:14:17.307646 19286 tablet_bootstrap.cc:654] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.318367 19283 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7: No bootstrap required, opened a new log
I20260709 12:14:17.319034 19283 ts_tablet_manager.cc:1403] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.012s
I20260709 12:14:17.322283 19283 raft_consensus.cc:359] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.329493 19284 raft_consensus.cc:359] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.330415 19284 raft_consensus.cc:385] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.330667 19284 raft_consensus.cc:740] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.331456 19284 consensus_queue.cc:260] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.333865 19284 ts_tablet_manager.cc:1434] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 12:14:17.336463 19283 raft_consensus.cc:385] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.336975 19283 raft_consensus.cc:740] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.337767 19283 consensus_queue.cc:260] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.339598 19283 ts_tablet_manager.cc:1434] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:14:17.342382 19286 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:17.343060 19286 ts_tablet_manager.cc:1403] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.004s
I20260709 12:14:17.351788 19284 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:17.360771 19284 tablet_bootstrap.cc:654] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.364678 19286 raft_consensus.cc:359] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.365938 19286 raft_consensus.cc:385] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.366258 19286 raft_consensus.cc:740] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.367180 19286 consensus_queue.cc:260] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.369858 19286 ts_tablet_manager.cc:1434] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:14:17.371193 19286 tablet_bootstrap.cc:492] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8: Bootstrap starting.
I20260709 12:14:17.371948 19782 raft_consensus.cc:515] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.378196 19286 tablet_bootstrap.cc:654] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.402510 19782 leader_election.cc:290] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.402765 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:17.380941 19088 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:17.401489 18911 catalog_manager.cc:5697] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:17.409731 19087 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.410274 19683 raft_consensus.cc:493] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.410902 19683 raft_consensus.cc:515] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.413344 19683 leader_election.cc:290] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:17.415920 19011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43f821d9236644bcb6676552b6ee4019" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
W20260709 12:14:17.417735 19203 leader_election.cc:343] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.423233 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43f821d9236644bcb6676552b6ee4019" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:17.423967 19164 raft_consensus.cc:2468] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:17.423888 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:17.426159 19015 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.428129 19203 leader_election.cc:304] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:17.429512 19683 raft_consensus.cc:2804] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:17.429912 19683 raft_consensus.cc:493] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.430258 19683 raft_consensus.cc:3060] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.436240 19015 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:17.441020 19129 leader_election.cc:304] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:17.442085 19782 raft_consensus.cc:2804] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:17.442952 19782 raft_consensus.cc:697] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:17.444592 19782 consensus_queue.cc:237] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.445093 19683 raft_consensus.cc:515] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.455631 19286 tablet_bootstrap.cc:492] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8: No bootstrap required, opened a new log
I20260709 12:14:17.456441 19286 ts_tablet_manager.cc:1403] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.010s
I20260709 12:14:17.456096 19680 consensus_queue.cc:1048] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:17.462203 19286 raft_consensus.cc:359] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:17.463778 19286 raft_consensus.cc:385] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.464085 19286 raft_consensus.cc:740] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.466401 19015 raft_consensus.cc:3060] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.464838 19286 consensus_queue.cc:260] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:17.470546 19286 ts_tablet_manager.cc:1434] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:14:17.473743 19087 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:17.477170 19284 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:17.477805 19284 ts_tablet_manager.cc:1403] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.143s	user 0.030s	sys 0.036s
I20260709 12:14:17.481554 19284 raft_consensus.cc:359] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.482334 19284 raft_consensus.cc:385] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.482371 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43f821d9236644bcb6676552b6ee4019" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:17.482671 19284 raft_consensus.cc:740] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.483208 19160 raft_consensus.cc:3060] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.483660 19284 consensus_queue.cc:260] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.486277 19284 ts_tablet_manager.cc:1434] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:14:17.488757 19284 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:17.499158 19284 tablet_bootstrap.cc:654] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.511788 19160 raft_consensus.cc:2468] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:17.513279 19203 leader_election.cc:304] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:17.518288 19680 raft_consensus.cc:2804] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:17.519229 19680 raft_consensus.cc:697] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:17.520304 19680 consensus_queue.cc:237] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.522212 19014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43f821d9236644bcb6676552b6ee4019" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
W20260709 12:14:17.525182 19203 leader_election.cc:343] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:17.520753 19683 leader_election.cc:290] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:17.528460 19284 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:17.529136 19284 ts_tablet_manager.cc:1403] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.019s
I20260709 12:14:17.533113 19284 raft_consensus.cc:359] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.533867 19284 raft_consensus.cc:385] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.534145 19284 raft_consensus.cc:740] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.534888 19284 consensus_queue.cc:260] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:17.535785 19203 consensus_peers.cc:597] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:17.537640 19284 ts_tablet_manager.cc:1434] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:14:17.539098 19284 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:17.545982 19284 tablet_bootstrap.cc:654] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.592279 19284 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:17.592962 19284 ts_tablet_manager.cc:1403] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.008s
I20260709 12:14:17.596427 19284 raft_consensus.cc:359] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.597388 19284 raft_consensus.cc:385] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.596643 18910 catalog_manager.cc:5697] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:17.597714 19284 raft_consensus.cc:740] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.600149 19284 consensus_queue.cc:260] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.610051 19284 ts_tablet_manager.cc:1434] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 12:14:17.611348 19284 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:17.630180 19239 raft_consensus.cc:3060] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.618439 18911 catalog_manager.cc:5697] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:17.635846 19284 tablet_bootstrap.cc:654] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:17.653769 19284 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: No bootstrap required, opened a new log
I20260709 12:14:17.654420 19284 ts_tablet_manager.cc:1403] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.011s
I20260709 12:14:17.657823 19284 raft_consensus.cc:359] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.658565 19284 raft_consensus.cc:385] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:17.658865 19284 raft_consensus.cc:740] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:17.659751 19284 consensus_queue.cc:260] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.661789 19284 ts_tablet_manager.cc:1434] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:14:17.685386 19707 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.686172 19707 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.688496 19707 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.690867 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:17.691680 19088 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:17.693346 19204 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:17.696269 19707 raft_consensus.cc:2804] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:17.690397 19656 consensus_queue.cc:1048] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:14:17.696717 19707 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.698439 19707 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.700620 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1" is_pre_election: true
I20260709 12:14:17.701421 19015 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 0.
I20260709 12:14:17.720459 19393 raft_consensus.cc:493] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.721222 19393 raft_consensus.cc:515] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:17.725845 19393 leader_election.cc:290] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:17.727651 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:17.744320 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:17.763576 18980 leader_election.cc:304] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:17.765311 19393 raft_consensus.cc:2749] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:17.813004 19014 raft_consensus.cc:3060] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.824594 19749 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.820566 19705 consensus_queue.cc:1048] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:14:17.837059 19707 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.834579 19749 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:17.847218 19707 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:17.847625 19749 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:17.849960 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:17.851478 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:17.851718 19015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
I20260709 12:14:17.855942 19091 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.855983 19015 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:17.865134 19782 consensus_queue.cc:1048] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:17.867918 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:17.879303 19015 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:17.880558 18980 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:17.880636 19203 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:17.886718 19595 raft_consensus.cc:2749] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:17.887391 19680 raft_consensus.cc:2804] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:17.888003 19680 raft_consensus.cc:697] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:17.888916 19680 consensus_queue.cc:237] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:17.898180 19091 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:17.905918 19705 consensus_queue.cc:1048] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:17.949118 18910 catalog_manager.cc:5697] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 1 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:17.979677 19782 consensus_queue.cc:1048] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:18.014217 19015 raft_consensus.cc:3060] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:18.024583 19782 consensus_queue.cc:1048] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:18.031028 19014 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:18.051167 19705 consensus_queue.cc:1048] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:18.099853 19645 consensus_queue.cc:1048] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:18.383961 19645 consensus_queue.cc:1048] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:18.397413 19680 consensus_queue.cc:1048] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:19.412122  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
W20260709 12:14:19.425506 19203 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:19.437577 19203 consensus_peers.cc:597] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.444190 19129 consensus_peers.cc:597] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.447568 19055 consensus_peers.cc:597] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.467002 19129 consensus_peers.cc:597] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.469050 19203 consensus_peers.cc:597] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.473013 19203 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.474859 19055 consensus_peers.cc:597] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.477824 19203 consensus_peers.cc:597] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.480875  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:14:19.482084 19129 consensus_peers.cc:597] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.482153  9261 tablet_replica.cc:333] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.483526  9261 raft_consensus.cc:2243] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.484263  9261 raft_consensus.cc:2272] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.486505  9261 tablet_replica.cc:333] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.487212  9261 raft_consensus.cc:2243] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.487609  9261 raft_consensus.cc:2272] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.490023  9261 tablet_replica.cc:333] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.490844  9261 raft_consensus.cc:2243] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.491387  9261 raft_consensus.cc:2272] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:19.493687 19203 consensus_peers.cc:597] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.494211  9261 tablet_replica.cc:333] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.495302  9261 raft_consensus.cc:2243] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.495813  9261 raft_consensus.cc:2272] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.497925  9261 tablet_replica.cc:333] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.498793  9261 raft_consensus.cc:2243] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.499348  9261 raft_consensus.cc:2272] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.501705  9261 tablet_replica.cc:333] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
W20260709 12:14:19.501792 19203 consensus_peers.cc:597] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.502460  9261 raft_consensus.cc:2243] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.502949  9261 raft_consensus.cc:2272] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.505577  9261 tablet_replica.cc:333] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.506289  9261 raft_consensus.cc:2243] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.506841  9261 raft_consensus.cc:2272] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.509059  9261 tablet_replica.cc:333] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.509655  9261 raft_consensus.cc:2243] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.510044  9261 raft_consensus.cc:2272] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.512200  9261 tablet_replica.cc:333] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.513139  9261 raft_consensus.cc:2243] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.513670  9261 raft_consensus.cc:2272] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.516242  9261 tablet_replica.cc:333] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.516953  9261 raft_consensus.cc:2243] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.517436  9261 raft_consensus.cc:2272] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.520105  9261 tablet_replica.cc:333] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.520982  9261 raft_consensus.cc:2243] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.521539  9261 raft_consensus.cc:2272] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:19.523924 19055 consensus_peers.cc:597] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.524434  9261 tablet_replica.cc:333] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
W20260709 12:14:19.524893 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.525331  9261 raft_consensus.cc:2243] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.525861  9261 raft_consensus.cc:2272] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.528060  9261 tablet_replica.cc:333] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.528810  9261 raft_consensus.cc:2243] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.529331  9261 raft_consensus.cc:2272] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.532204  9261 tablet_replica.cc:333] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.533018  9261 raft_consensus.cc:2243] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.534101  9261 raft_consensus.cc:2272] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.536701  9261 tablet_replica.cc:333] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.537534  9261 raft_consensus.cc:2243] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.538643  9261 raft_consensus.cc:2272] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.541765  9261 tablet_replica.cc:333] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.546356  9261 raft_consensus.cc:2243] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.547096  9261 raft_consensus.cc:2272] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:19.547878 19055 consensus_peers.cc:597] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.548748 19129 consensus_peers.cc:597] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.550069  9261 tablet_replica.cc:333] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.550854  9261 raft_consensus.cc:2243] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.551578  9261 raft_consensus.cc:2272] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.555094  9261 tablet_replica.cc:333] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.556102  9261 raft_consensus.cc:2243] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.556607  9261 raft_consensus.cc:2272] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.559161  9261 tablet_replica.cc:333] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.559829  9261 raft_consensus.cc:2243] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.560312  9261 raft_consensus.cc:2272] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.562965  9261 tablet_replica.cc:333] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.563971  9261 raft_consensus.cc:2243] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.564990  9261 raft_consensus.cc:2272] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.568162  9261 tablet_replica.cc:333] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.568841  9261 raft_consensus.cc:2243] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.569763  9261 raft_consensus.cc:2272] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.572062  9261 tablet_replica.cc:333] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.573309  9261 raft_consensus.cc:2243] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.573884  9261 raft_consensus.cc:2272] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.577032  9261 tablet_replica.cc:333] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.577950  9261 raft_consensus.cc:2243] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.578567  9261 raft_consensus.cc:2272] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.581864  9261 tablet_replica.cc:333] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.582787  9261 raft_consensus.cc:2243] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.583437  9261 raft_consensus.cc:2272] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.586671  9261 tablet_replica.cc:333] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.587603  9261 raft_consensus.cc:2243] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.588109  9261 raft_consensus.cc:2272] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.590332  9261 tablet_replica.cc:333] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.590953  9261 raft_consensus.cc:2243] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.591979  9261 raft_consensus.cc:2272] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.594290  9261 tablet_replica.cc:333] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.595031  9261 raft_consensus.cc:2243] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.595528  9261 raft_consensus.cc:2272] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.597972  9261 tablet_replica.cc:333] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.598573  9261 raft_consensus.cc:2243] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.599143  9261 raft_consensus.cc:2272] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.602456  9261 tablet_replica.cc:333] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.603358  9261 raft_consensus.cc:2243] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.603809  9261 raft_consensus.cc:2272] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.606143  9261 tablet_replica.cc:333] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.606822  9261 raft_consensus.cc:2243] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.608006  9261 raft_consensus.cc:2272] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.610445  9261 tablet_replica.cc:333] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.611198  9261 raft_consensus.cc:2243] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.611732  9261 raft_consensus.cc:2272] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.614761  9261 tablet_replica.cc:333] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.615681  9261 raft_consensus.cc:2243] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.616894  9261 raft_consensus.cc:2272] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.619973  9261 tablet_replica.cc:333] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.620852  9261 raft_consensus.cc:2243] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.621434  9261 raft_consensus.cc:2272] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.624682  9261 tablet_replica.cc:333] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.625633  9261 raft_consensus.cc:2243] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.626888  9261 raft_consensus.cc:2272] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.630296  9261 tablet_replica.cc:333] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.631163  9261 raft_consensus.cc:2243] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:19.631234 19203 consensus_peers.cc:597] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.632043  9261 raft_consensus.cc:2272] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:19.632121 19203 consensus_peers.cc:597] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.635584  9261 tablet_replica.cc:333] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
W20260709 12:14:19.636047 19129 consensus_peers.cc:597] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.636523  9261 raft_consensus.cc:2243] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.637276  9261 raft_consensus.cc:2272] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.639883  9261 tablet_replica.cc:333] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.640682  9261 raft_consensus.cc:2243] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.641244  9261 raft_consensus.cc:2272] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.643822  9261 tablet_replica.cc:333] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.644631  9261 raft_consensus.cc:2243] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.645743  9261 raft_consensus.cc:2272] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:19.647228 19055 consensus_peers.cc:597] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.648471  9261 tablet_replica.cc:333] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.649179  9261 raft_consensus.cc:2243] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.650120  9261 raft_consensus.cc:2272] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.652505  9261 tablet_replica.cc:333] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.653275  9261 raft_consensus.cc:2243] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.654486  9261 raft_consensus.cc:2272] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.657469  9261 tablet_replica.cc:333] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
W20260709 12:14:19.657832 19203 consensus_peers.cc:597] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.658263  9261 raft_consensus.cc:2243] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.658774  9261 raft_consensus.cc:2272] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.661412  9261 tablet_replica.cc:333] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.662060  9261 raft_consensus.cc:2243] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.662601  9261 raft_consensus.cc:2272] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:19.665288 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.666000  9261 tablet_replica.cc:333] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.666920  9261 raft_consensus.cc:2243] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:19.667899  9261 raft_consensus.cc:2272] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.670141  9261 tablet_replica.cc:333] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
I20260709 12:14:19.670969  9261 raft_consensus.cc:2243] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.671553  9261 raft_consensus.cc:2272] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.674830  9261 tablet_replica.cc:333] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: stopping tablet replica
W20260709 12:14:19.675122 19129 consensus_peers.cc:597] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.675554  9261 raft_consensus.cc:2243] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:19.676268  9261 raft_consensus.cc:2272] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:19.748080  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
W20260709 12:14:19.781824 19203 consensus_peers.cc:597] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.788322 19203 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.788884 19203 consensus_peers.cc:597] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.797392 19129 consensus_peers.cc:597] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.800634 19203 consensus_peers.cc:597] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.822118 19129 consensus_peers.cc:597] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.833518 19055 consensus_peers.cc:597] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.842058 19055 consensus_peers.cc:597] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:19.849071 19129 consensus_peers.cc:597] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.865267  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:19.873806 19828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:19.873845 19827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:19.877311 19830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:19.877369  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:19.883090  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:19.883447  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:19.883695  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599259883673 us; error 0 us; skew 500 ppm
I20260709 12:14:19.884509  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:19.888484  9261 webserver.cc:533] Webserver started at http://127.9.11.65:40003/ using document root <none> and password file <none>
I20260709 12:14:19.889213  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:19.889487  9261 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:14:19.895370  9261 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:14:19.907044 19835 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:19.908138  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:14:19.908516  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "3fcac1aec8df4c599b4c67550ce347e1"
format_stamp: "Formatted at 2026-07-09 12:14:04 on dist-test-slave-k289"
I20260709 12:14:19.909026  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:14:19.922338 19055 consensus_peers.cc:597] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:19.923892  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:19.925482  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:19.927714  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:19.935528 19842 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:14:19.954625 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:20.303364  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:14:20.303905  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.371s	user 0.010s	sys 0.001s
I20260709 12:14:20.304301  9261 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:14:20.311343 19842 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.324927 19842 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.325865 19842 tablet_bootstrap.cc:492] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.326540 19842 ts_tablet_manager.cc:1403] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:14:20.329411 19842 raft_consensus.cc:359] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.330113 19842 raft_consensus.cc:740] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.330890 19842 consensus_queue.cc:260] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.338892 19842 ts_tablet_manager.cc:1434] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.008s
I20260709 12:14:20.339699 19842 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.355048 19842 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.356125 19842 tablet_bootstrap.cc:492] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.356755 19842 ts_tablet_manager.cc:1403] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.002s
I20260709 12:14:20.358740 19842 raft_consensus.cc:359] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.359364 19842 raft_consensus.cc:740] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.359899 19842 consensus_queue.cc:260] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.361234 19842 ts_tablet_manager.cc:1434] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:14:20.361913 19842 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.376984 19842 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.378046 19842 tablet_bootstrap.cc:492] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.378749 19842 ts_tablet_manager.cc:1403] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 12:14:20.391326 19842 raft_consensus.cc:359] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.392127 19842 raft_consensus.cc:740] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.392846 19842 consensus_queue.cc:260] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.394903 19842 ts_tablet_manager.cc:1434] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.000s
I20260709 12:14:20.401263 19842 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.430016 19842 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.447638 19842 tablet_bootstrap.cc:492] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.448462 19842 ts_tablet_manager.cc:1403] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.004s
I20260709 12:14:20.454095 19842 raft_consensus.cc:359] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.454845 19842 raft_consensus.cc:740] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.455921 19842 consensus_queue.cc:260] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.458009 19842 ts_tablet_manager.cc:1434] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:14:20.458827 19842 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.471654 19842 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.472575 19842 tablet_bootstrap.cc:492] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.473302 19842 ts_tablet_manager.cc:1403] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 12:14:20.476110 19842 raft_consensus.cc:359] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.476691 19842 raft_consensus.cc:740] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.477558 19842 consensus_queue.cc:260] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.479307 19842 ts_tablet_manager.cc:1434] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:14:20.479996 19842 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.494359 19842 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.495383 19842 tablet_bootstrap.cc:492] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.496114 19842 ts_tablet_manager.cc:1403] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:14:20.498843 19842 raft_consensus.cc:359] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.499648 19842 raft_consensus.cc:740] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.500375 19842 consensus_queue.cc:260] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.502466 19842 ts_tablet_manager.cc:1434] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:14:20.503278 19842 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.505463 19803 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.506130 19803 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.508258 19803 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:20.510802 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
W20260709 12:14:20.519358 19055 leader_election.cc:336] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.519917 19055 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:20.522217 19804 raft_consensus.cc:2749] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.526520 19842 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.527554 19842 tablet_bootstrap.cc:492] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.528180 19842 ts_tablet_manager.cc:1403] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.012s
I20260709 12:14:20.530320 19842 raft_consensus.cc:359] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.531028 19842 raft_consensus.cc:740] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.531816 19842 consensus_queue.cc:260] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.533769 19842 ts_tablet_manager.cc:1434] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:14:20.534566 19842 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.550383 19842 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.551473 19842 tablet_bootstrap.cc:492] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.552205 19842 ts_tablet_manager.cc:1403] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:14:20.553591  9261 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:14:20.554021  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.250s	user 0.241s	sys 0.005s
I20260709 12:14:20.555089 19842 raft_consensus.cc:359] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.555962 19842 raft_consensus.cc:740] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.556877 19842 consensus_queue.cc:260] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.559098 19842 ts_tablet_manager.cc:1434] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:20.559991 19842 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.581748 19842 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.583235 19842 tablet_bootstrap.cc:492] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.584189 19842 ts_tablet_manager.cc:1403] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.006s
I20260709 12:14:20.589105 19842 raft_consensus.cc:359] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.590116 19842 raft_consensus.cc:740] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.591259 19842 consensus_queue.cc:260] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.594750 19842 ts_tablet_manager.cc:1434] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:14:20.595875 19842 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.616938 19804 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.617688 19804 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.623224 19804 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:20.624377 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
W20260709 12:14:20.628433 19055 leader_election.cc:336] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.628906 19055 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:20.630200 19803 raft_consensus.cc:2749] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.638305 19842 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.639732 19842 tablet_bootstrap.cc:492] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.640700 19842 ts_tablet_manager.cc:1403] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.007s
I20260709 12:14:20.644439 19842 raft_consensus.cc:359] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.645375 19842 raft_consensus.cc:740] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.646565 19842 consensus_queue.cc:260] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.649636 19842 ts_tablet_manager.cc:1434] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 12:14:20.650841 19842 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.671154 19822 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.671859 19822 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.674378 19822 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.676575 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:20.679694 19842 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.681128 19842 tablet_bootstrap.cc:492] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.682106 19842 ts_tablet_manager.cc:1403] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.008s
I20260709 12:14:20.682842 19822 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.683476 19822 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.685678 19822 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:20.686537 19203 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.686182 19842 raft_consensus.cc:359] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.687076 19203 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:20.687252 19842 raft_consensus.cc:740] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.687875 19822 raft_consensus.cc:2749] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.689414 19842 consensus_queue.cc:260] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.692782 19842 ts_tablet_manager.cc:1434] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:14:20.693984 19842 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.694579 19822 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.695254 19822 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.696484 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:20.697646 19822 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.698894 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:20.706348 19850 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.706950 19850 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.719272 19842 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.720438 19842 tablet_bootstrap.cc:492] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.721159 19842 ts_tablet_manager.cc:1403] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.007s
W20260709 12:14:20.723426 19129 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.724155 19842 raft_consensus.cc:359] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.724964 19842 raft_consensus.cc:740] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.725934 19842 consensus_queue.cc:260] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.726511 19850 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.728461 19842 ts_tablet_manager.cc:1434] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:14:20.729440 19842 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:20.731240 19203 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.731858 19203 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:20.734638 19822 raft_consensus.cc:2749] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:20.736019 19203 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.736676 19203 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:20.737512 19822 raft_consensus.cc:2749] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.743912 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:20.744582 19239 raft_consensus.cc:2468] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:20.748553 19850 raft_consensus.cc:493] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.749190 19850 raft_consensus.cc:515] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.751116 19842 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.751906 19130 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.752513 19842 tablet_bootstrap.cc:492] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.753549 19842 ts_tablet_manager.cc:1403] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 12:14:20.753906 19847 raft_consensus.cc:2804] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:20.754371 19847 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.754773 19847 raft_consensus.cc:3060] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.757493 19842 raft_consensus.cc:359] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.758549 19842 raft_consensus.cc:740] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.759516 19842 consensus_queue.cc:260] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.760939 19822 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.761488 19822 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:20.763964 19842 ts_tablet_manager.cc:1434] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
W20260709 12:14:20.764827 19129 leader_election.cc:336] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.765239 19842 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.765811 19850 leader_election.cc:290] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.772208 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:20.778898 19822 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.783998 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:20.788519 19842 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.781492 19847 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.789925 19842 tablet_bootstrap.cc:492] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.790947 19842 ts_tablet_manager.cc:1403] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 12:14:20.792730 19804 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.793368 19804 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.794834 19842 raft_consensus.cc:359] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.795687 19804 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:20.795905 19842 raft_consensus.cc:740] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
W20260709 12:14:20.796777 19129 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.796957 19842 consensus_queue.cc:260] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.797879 19847 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.800334 19842 ts_tablet_manager.cc:1434] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:14:20.800483 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:20.801895 19842 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.803695 19130 leader_election.cc:304] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.807811 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ff6615d74da465f9105797edb9c3698" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
W20260709 12:14:20.808125 19203 leader_election.cc:336] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.808461 19239 raft_consensus.cc:3060] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.808863 19203 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:20.809911 19822 raft_consensus.cc:2749] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.811167 19850 raft_consensus.cc:2749] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:20.815510 19055 leader_election.cc:336] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.816820 19055 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:20.821774 19239 raft_consensus.cc:2468] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:20.822067 19804 raft_consensus.cc:2749] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.823628 19130 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.824960 19850 raft_consensus.cc:2804] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:20.825568 19850 raft_consensus.cc:697] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:20.826548 19850 consensus_queue.cc:237] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.830672 19705 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.831303 19705 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.832393 19842 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.833447 19705 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.833968 19842 tablet_bootstrap.cc:492] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.835130 19842 ts_tablet_manager.cc:1403] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 12:14:20.838798 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:20.839035 19842 raft_consensus.cc:359] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.840188 19842 raft_consensus.cc:740] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.841272 19842 consensus_queue.cc:260] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.843372 19656 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.843933 19656 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.844774 19842 ts_tablet_manager.cc:1434] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:14:20.846120 19842 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.850481 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:20.851240 19238 raft_consensus.cc:2468] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:20.854676 19656 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.856724 19130 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:20.858394 19656 raft_consensus.cc:2804] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:20.858764 19656 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.859119 19656 raft_consensus.cc:3060] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:20.864028 19203 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.864542 19203 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:20.866329 19705 raft_consensus.cc:2749] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:20.868801 19129 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.870539 19804 raft_consensus.cc:493] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.872277 19804 raft_consensus.cc:515] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.877050 19803 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.877849 19803 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.880004 19803 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.881696 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:20.883150 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:20.885799 19804 leader_election.cc:290] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.902760 19850 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.903756 19850 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.906437 19842 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:20.907994 19842 tablet_bootstrap.cc:492] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:20.907909 19847 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.908767 19847 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.911717 19847 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.893498 19656 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.912863 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:20.913614 19091 raft_consensus.cc:2468] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:20.914551 19656 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:20.918274 19055 leader_election.cc:336] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.918795 19055 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.919159 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:20.919701 19845 raft_consensus.cc:2749] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:20.921062 19055 leader_election.cc:336] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.919824 19239 raft_consensus.cc:3060] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.921497 19055 leader_election.cc:304] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.922283 19914 raft_consensus.cc:2749] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:20.916025 19130 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
W20260709 12:14:20.924829 19129 leader_election.cc:336] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.924949 19705 raft_consensus.cc:493] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.926110 19847 raft_consensus.cc:2804] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:20.926635 19847 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.928498 19842 ts_tablet_manager.cc:1403] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.083s	user 0.031s	sys 0.019s
I20260709 12:14:20.928969 19847 raft_consensus.cc:3060] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.930040 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:20.931111 19091 raft_consensus.cc:2468] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:20.935859 19842 raft_consensus.cc:359] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.936842 19842 raft_consensus.cc:740] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:20.937772 19842 consensus_queue.cc:260] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:20.927179 19129 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.925683 19705 raft_consensus.cc:515] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.928397 19850 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.938555 19847 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.944811 19842 ts_tablet_manager.cc:1434] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:14:20.944901 19705 leader_election.cc:290] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.945915 19842 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:20.946096 19847 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.938555 19130 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
W20260709 12:14:20.955720 19203 leader_election.cc:336] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.953998 18911 catalog_manager.cc:5697] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:20.957198 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:20.957649 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:20.958127 19091 raft_consensus.cc:3060] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.958575 19163 raft_consensus.cc:2468] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:20.959887 19203 leader_election.cc:304] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.959935 19845 raft_consensus.cc:493] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.960680 19705 raft_consensus.cc:2804] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:20.961301 19705 raft_consensus.cc:493] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.961827 19705 raft_consensus.cc:3060] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.963222 19239 raft_consensus.cc:2468] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:20.964176 19847 raft_consensus.cc:2804] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:20.964630 19847 raft_consensus.cc:493] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.965104 19847 raft_consensus.cc:3060] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.964888 19130 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:20.965961 19656 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.960665 19845 raft_consensus.cc:515] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:20.966495 19656 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:20.968935 19656 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:20.969504 19656 raft_consensus.cc:2804] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:20.969990 19656 raft_consensus.cc:697] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:20.970788 19656 consensus_queue.cc:237] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:20.974943 19055 leader_election.cc:336] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.969426 19845 leader_election.cc:290] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:20.976423 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:20.977293 19087 raft_consensus.cc:2468] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
I20260709 12:14:20.983036 19130 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:20.989701 19850 raft_consensus.cc:2804] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:20.990273 19850 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:20.990772 19850 raft_consensus.cc:3060] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:20.992563 19705 raft_consensus.cc:515] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:20.994678 19705 leader_election.cc:290] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:20.997505 19129 leader_election.cc:336] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:20.999991 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:21.015481 19845 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
W20260709 12:14:21.048838 19129 leader_election.cc:336] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.036511 19847 raft_consensus.cc:515] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.041978 19850 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
W20260709 12:14:21.043824 19203 leader_election.cc:336] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.011783 19091 raft_consensus.cc:2468] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:21.056823 19055 leader_election.cc:304] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:21.055075 19845 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:21.057987 19656 raft_consensus.cc:493] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
W20260709 12:14:21.061873 19129 leader_election.cc:336] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.027858 19804 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.066025 19847 leader_election.cc:290] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.065965 19804 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.067373 19803 raft_consensus.cc:2749] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:21.069757 19845 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:21.070508 19850 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.063856 19656 raft_consensus.cc:515] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.073391 19656 leader_election.cc:290] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:21.076043 19129 leader_election.cc:336] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.076104 19804 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:21.081645 19705 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.082476 19705 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:21.083570 19129 leader_election.cc:336] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:21.084801 19129 leader_election.cc:336] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.087292 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:21.088109 19091 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:21.089184 19822 raft_consensus.cc:493] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.089967 19822 raft_consensus.cc:515] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:21.091832 19203 leader_election.cc:336] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.096518 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:21.097265 19238 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:21.097517 19804 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.098146 19804 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.099782 19204 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:21.100209 19842 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:21.100355 19804 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:21.101438 19842 tablet_bootstrap.cc:492] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:21.102227 19842 ts_tablet_manager.cc:1403] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.157s	user 0.016s	sys 0.008s
I20260709 12:14:21.102928 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:21.103865 19238 raft_consensus.cc:2468] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:21.105304 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:21.106047 19056 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.106156 19091 raft_consensus.cc:2468] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 1.
I20260709 12:14:21.106904 19803 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:21.107427 19803 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.107827 19803 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.108075 19204 leader_election.cc:304] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.109577 19680 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:21.110105 19680 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.110549 19680 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:21.111963 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 132742us
I20260709 12:14:21.113118 19705 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.115561 19705 raft_consensus.cc:2804] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:21.116487 19705 raft_consensus.cc:493] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.117019 19705 raft_consensus.cc:3060] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.120083 19842 raft_consensus.cc:359] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.120889 19842 raft_consensus.cc:740] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:21.123761 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:21.124543 19238 raft_consensus.cc:2468] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 1.
I20260709 12:14:21.125113 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1280ae7e994dc59cf16f8ddef18b72" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:21.125005 19842 consensus_queue.cc:260] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.125727 19163 raft_consensus.cc:3060] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.128916 19842 ts_tablet_manager.cc:1434] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 12:14:21.129832 19842 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:21.135767 19203 leader_election.cc:336] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.139689 19056 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.138619 19163 raft_consensus.cc:2468] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:21.152204 19922 raft_consensus.cc:2804] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:21.152695 19922 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.153118 19922 raft_consensus.cc:3060] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.149852 19130 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:21.163869 19850 raft_consensus.cc:2804] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.163671 19803 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:21.164558 19203 leader_election.cc:304] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:21.164597 19850 raft_consensus.cc:697] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:21.116079 19822 leader_election.cc:290] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:21.174593 19055 leader_election.cc:336] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.156091 19056 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.176141 19680 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.185612 19803 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:21.187462 19680 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.190683 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ddebbef4404377b6b25cc37866b9e2" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:21.191618 19091 raft_consensus.cc:3060] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.200399 19705 raft_consensus.cc:515] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.206627 19928 raft_consensus.cc:2804] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.207587 19928 raft_consensus.cc:697] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:21.215408 19705 leader_election.cc:290] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.222693 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:21.223639 19087 raft_consensus.cc:3060] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:21.225229 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 44134.9us
I20260709 12:14:21.188349 19914 raft_consensus.cc:2804] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:21.238880 19091 raft_consensus.cc:2468] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:21.240823 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:21.241668 19088 raft_consensus.cc:2468] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 1.
W20260709 12:14:21.246657 19204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 138703us
I20260709 12:14:21.248358 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:21.248991 19091 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.251263 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:21.251926 19239 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:21.252803 19055 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:21.253835 19055 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.263370 19239 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:21.266206 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:21.266965 19090 raft_consensus.cc:3060] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.267208 19928 consensus_queue.cc:237] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.236943 19914 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.276471 19914 raft_consensus.cc:3060] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.277235 19087 raft_consensus.cc:2468] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:21.278147 19842 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:21.279274 19842 tablet_bootstrap.cc:492] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:21.224522 19850 consensus_queue.cc:237] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.280001 19842 ts_tablet_manager.cc:1403] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.150s	user 0.026s	sys 0.011s
I20260709 12:14:21.287393 19842 raft_consensus.cc:359] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.288414 19842 raft_consensus.cc:740] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:21.289511 19842 consensus_queue.cc:260] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
W20260709 12:14:21.293805 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 50632.8us
I20260709 12:14:21.298897 19091 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:21.305123 19922 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.306706 19204 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.310724 19842 ts_tablet_manager.cc:1434] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 12:14:21.315586 19842 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:21.322516 19129 consensus_peers.cc:597] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:21.323812 19090 raft_consensus.cc:2468] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:21.327565 19204 leader_election.cc:304] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.329545 19927 raft_consensus.cc:2804] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.330147 19927 raft_consensus.cc:697] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:21.331686 19927 consensus_queue.cc:237] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.332485 19922 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:21.324015 19933 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.334159 19933 raft_consensus.cc:697] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:21.336400 19933 consensus_queue.cc:237] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:21.340075 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 83636.3us
I20260709 12:14:21.343731 19056 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.347286 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:21.348160 19236 raft_consensus.cc:3060] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:21.350723 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 51993.7us
I20260709 12:14:21.355422 19922 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.346809 19914 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.359331 19914 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:21.367633 19236 raft_consensus.cc:2468] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:21.368049 19922 raft_consensus.cc:697] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:21.356078 19130 leader_election.cc:304] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:21.369903 19935 raft_consensus.cc:2804] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.370572 19935 raft_consensus.cc:697] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:21.371624 19935 consensus_queue.cc:237] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.372174 19922 consensus_queue.cc:237] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:21.377966 19130 leader_election.cc:304] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
W20260709 12:14:21.382345 19203 leader_election.cc:336] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.423353 19850 raft_consensus.cc:2804] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:14:21.424052 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 46236.1us
I20260709 12:14:21.420115 19056 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.457468 19850 raft_consensus.cc:493] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:21.458252 19850 raft_consensus.cc:3060] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:21.459345 19130 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:21.465081 19847 raft_consensus.cc:2804] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.465759 19847 raft_consensus.cc:697] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:21.449422 18911 catalog_manager.cc:5697] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 2 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:21.470924 18908 catalog_manager.cc:5697] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:21.466917 19847 consensus_queue.cc:237] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.473229 18908 catalog_manager.cc:5697] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:21.476301 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 56320.5us
W20260709 12:14:21.478193 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 95880.7us
I20260709 12:14:21.479482 18908 catalog_manager.cc:5697] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:21.482223 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d311e304cfc84db5a4dfc30459f4d72e" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:21.482893 19239 raft_consensus.cc:3060] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:21.480247 19203 leader_election.cc:336] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.493854 19938 raft_consensus.cc:2804] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.494637 19938 raft_consensus.cc:697] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:21.495636 19239 raft_consensus.cc:2468] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:21.495666 19938 consensus_queue.cc:237] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.470311 19850 raft_consensus.cc:515] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.501412 19850 leader_election.cc:290] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.504580 19056 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:21.510938 19914 raft_consensus.cc:2804] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:21.511559 19914 raft_consensus.cc:697] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:21.512481 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a4de1a07654315ae7678c1f9e8fcca" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:21.512518 19914 consensus_queue.cc:237] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.513382 19090 raft_consensus.cc:3060] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:21.521026 19129 consensus_peers.cc:597] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:21.537137 19090 raft_consensus.cc:2468] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
W20260709 12:14:21.560895 19055 leader_election.cc:336] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.565747 19130 leader_election.cc:304] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:21.566701 19935 raft_consensus.cc:2804] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:14:21.588696 19129 leader_election.cc:336] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.594132 19935 raft_consensus.cc:697] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:21.593642 18910 catalog_manager.cc:5697] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:21.595305 19935 consensus_queue.cc:237] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.609921 18911 catalog_manager.cc:5697] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:21.610055 18908 catalog_manager.cc:5697] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 2 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:21.623759 19234 raft_consensus.cc:1275] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.625448 19234 raft_consensus.cc:1275] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.632754 18910 catalog_manager.cc:5697] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:21.632350 18908 catalog_manager.cc:5697] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 2 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:21.640859 19935 consensus_queue.cc:1048] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:21.641883 19847 consensus_queue.cc:1048] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:21.634640 18910 catalog_manager.cc:5697] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:21.652529 19842 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:21.655656 19842 tablet_bootstrap.cc:492] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:21.656505 19842 ts_tablet_manager.cc:1403] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.341s	user 0.029s	sys 0.049s
W20260709 12:14:21.673836 19055 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:21.695088 19055 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:21.695305 19852 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.699592 19852 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.678776 19842 raft_consensus.cc:359] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.704006 19842 raft_consensus.cc:740] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:21.692472 18911 catalog_manager.cc:5697] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:21.712471 19842 consensus_queue.cc:260] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.704785 19852 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:21.771835 19842 ts_tablet_manager.cc:1434] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.115s	user 0.010s	sys 0.007s
I20260709 12:14:21.775913 19842 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:21.807235 19963 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.807984 19963 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:21.826200 19090 raft_consensus.cc:1275] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.828815 19161 raft_consensus.cc:1275] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.838546 19963 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:21.840512 19936 consensus_queue.cc:1048] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:21.847970 19951 consensus_queue.cc:1048] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:21.876490 19963 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.877218 19963 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.882473 19963 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:21.894712 19970 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:43327 every 8 connection(s)
I20260709 12:14:21.895234  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:43327
I20260709 12:14:21.901149 19161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:21.901441 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:21.910441 19842 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:21.933624 19842 tablet_bootstrap.cc:492] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:21.939409 19842 ts_tablet_manager.cc:1403] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.013s
I20260709 12:14:21.942351 19842 raft_consensus.cc:359] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.948163 19090 raft_consensus.cc:1275] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.954660 19088 raft_consensus.cc:1275] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.955024 19842 raft_consensus.cc:740] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:21.953991 19087 raft_consensus.cc:1275] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.956661 19847 consensus_queue.cc:1048] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:21.957165 19963 raft_consensus.cc:493] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.957711 19963 raft_consensus.cc:515] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.959219 19842 consensus_queue.cc:260] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:21.960079 19963 leader_election.cc:290] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:21.962323 19842 ts_tablet_manager.cc:1434] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.023s	user 0.001s	sys 0.005s
I20260709 12:14:21.962896 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:21.964901 19963 raft_consensus.cc:493] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.965436 19963 raft_consensus.cc:515] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:21.966713 19842 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:21.967940 19963 leader_election.cc:290] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:21.969245 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:21.969367 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:21.970067 19239 raft_consensus.cc:2393] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3fcac1aec8df4c599b4c67550ce347e1 in current term 2: Already voted for candidate 666fef4f7e5a4c32813dad8dfade8561 in this term.
I20260709 12:14:21.971421 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:21.971493 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:21.979207 19838 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:21.980988 19963 raft_consensus.cc:2749] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:21.981101 19838 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:21.983155 19963 raft_consensus.cc:2749] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:21.985760 19238 raft_consensus.cc:1275] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:21.987993 19932 consensus_queue.cc:1048] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:21.990674 19933 consensus_queue.cc:1048] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:21.993570 19933 consensus_queue.cc:1048] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:22.013702 19852 raft_consensus.cc:493] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.014670 19852 raft_consensus.cc:515] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.018287 19852 leader_election.cc:290] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:22.020195 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.025825 19088 raft_consensus.cc:1275] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.032591 19088 raft_consensus.cc:1275] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.032370 19847 consensus_queue.cc:1048] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:22.039453 19847 consensus_queue.cc:1048] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:22.049168 19239 raft_consensus.cc:1275] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.052300 19852 raft_consensus.cc:493] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.053354 19852 raft_consensus.cc:515] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.065433 19963 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.071650 19963 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:22.078277 19852 leader_election.cc:290] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:22.065062 19938 consensus_queue.cc:1048] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:14:22.096158 19161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.099077 19238 raft_consensus.cc:1275] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.113570 19938 consensus_queue.cc:1048] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:14:22.119933 19986 log.cc:927] Time spent T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:14:22.139398 19963 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:22.163916 19963 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.164549 19963 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.167411 19963 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:22.170270 19963 raft_consensus.cc:493] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.170802 19963 raft_consensus.cc:515] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.173563 19963 leader_election.cc:290] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:22.174543 19842 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:22.175820 19842 tablet_bootstrap.cc:492] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:22.176632 19842 ts_tablet_manager.cc:1403] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.210s	user 0.032s	sys 0.058s
I20260709 12:14:22.181792 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.183699 19842 raft_consensus.cc:359] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.184598 19842 raft_consensus.cc:740] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:22.185009 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.185350 19842 consensus_queue.cc:260] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.188187 19842 ts_tablet_manager.cc:1434] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:14:22.189124 19842 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:22.210271 19892 raft_consensus.cc:3060] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:22.238854 19889 raft_consensus.cc:3060] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:22.239322 19893 raft_consensus.cc:3060] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:22.239863 19890 raft_consensus.cc:3060] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:22.252100 19055 consensus_peers.cc:597] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:22.260166 19129 consensus_peers.cc:597] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:22.261559 19129 consensus_peers.cc:597] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.262867 19129 consensus_peers.cc:597] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:22.267500 19893 raft_consensus.cc:1275] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.268698 19852 raft_consensus.cc:493] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.269951 19055 consensus_peers.cc:597] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:22.283740 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:22.286302 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a3cd9e4dbb041ddb6775aa9ff22575b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:22.296761 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6079f38db31043d2bc846667f08d7d1b" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:22.309544 19055 consensus_peers.cc:597] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.280135 19852 raft_consensus.cc:515] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.335841 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "428f9c2176004b6aa007c6a7af377fcb" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.347890 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4c556a691543529aee3c07a4015d8d" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.348166 19994 raft_consensus.cc:493] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.348851 19994 raft_consensus.cc:515] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:22.351116 19994 leader_election.cc:290] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:22.351590 19994 raft_consensus.cc:493] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.357604 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eed4a045f6747428301e56cf20644b7" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.376209 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3afa3fafa244643a6da11e31e841d34" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.358213 19994 raft_consensus.cc:515] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.393924 19994 leader_election.cc:290] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:22.403033 19852 leader_election.cc:290] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
W20260709 12:14:22.405004 19992 log.cc:927] Time spent T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:14:22.406023 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04add48cb044473fbc449788540f24c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.433333 19892 raft_consensus.cc:1275] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.434640 19963 raft_consensus.cc:493] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.435228 19963 raft_consensus.cc:515] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.436249 19893 raft_consensus.cc:3060] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:22.437678 19890 raft_consensus.cc:1275] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.440917 19963 leader_election.cc:290] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:22.443934 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.449684 19842 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:22.450886 19842 tablet_bootstrap.cc:492] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:22.453192 19842 ts_tablet_manager.cc:1403] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.264s	user 0.028s	sys 0.000s
I20260709 12:14:22.456094 19842 raft_consensus.cc:359] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.456808 19842 raft_consensus.cc:740] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:22.466035 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:22.467041 19203 consensus_peers.cc:597] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:22.469204 19203 consensus_peers.cc:597] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.473824 19203 consensus_peers.cc:597] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:22.475871 19203 consensus_peers.cc:597] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.476858 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.476620 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a717f41e3045c8b600ca072c4b1efd" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.478695 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:22.480670 19924 consensus_queue.cc:1048] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:22.480995 19850 consensus_queue.cc:1048] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:22.483112 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:22.486251 19129 consensus_peers.cc:597] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.487028 19129 consensus_peers.cc:597] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.488252 19161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.489143 19842 consensus_queue.cc:260] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.493266 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
W20260709 12:14:22.495513 19203 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.497363 19838 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.488394 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a377e1ad0954438589b509e2491c8a72" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.498903 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8e8cd17ef794dd18367b9052aae4c29" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:22.499832 19842 ts_tablet_manager.cc:1434] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:14:22.508138 19842 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:22.504370 19838 leader_election.cc:304] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.540964 19963 raft_consensus.cc:2749] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:22.553696 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 140672us
I20260709 12:14:22.538928 19893 raft_consensus.cc:1275] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:22.541234 19838 leader_election.cc:304] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.556180 19994 raft_consensus.cc:2749] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.575111 19999 consensus_queue.cc:1048] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:22.599090 19998 raft_consensus.cc:2749] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:22.619797 19838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 79093us
I20260709 12:14:22.624501 19838 leader_election.cc:304] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7
W20260709 12:14:22.647878 19055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:32851, user_credentials={real_user=slave}} blocked reactor thread for 74867.8us
I20260709 12:14:22.641198 19994 raft_consensus.cc:2749] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.657892 19963 raft_consensus.cc:493] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.662524 19963 raft_consensus.cc:515] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:22.665056 19963 leader_election.cc:290] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:22.673679 19838 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:22.675426 19838 leader_election.cc:304] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.677340 19994 raft_consensus.cc:2749] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.692582 19838 leader_election.cc:304] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.693565 19994 raft_consensus.cc:2749] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.694911 19838 leader_election.cc:304] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.696029 19994 raft_consensus.cc:2749] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.687533 19963 raft_consensus.cc:2749] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.697734 19838 leader_election.cc:304] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.698966 19963 raft_consensus.cc:2749] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.701341 19838 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:22.701790 19842 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:22.703091 19842 tablet_bootstrap.cc:492] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:22.703377 19963 raft_consensus.cc:2749] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.703823 19842 ts_tablet_manager.cc:1403] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.196s	user 0.016s	sys 0.011s
I20260709 12:14:22.706887 19842 raft_consensus.cc:359] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.707654 19842 raft_consensus.cc:740] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:22.708401 19842 consensus_queue.cc:260] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:22.710688 19842 ts_tablet_manager.cc:1434] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:14:22.711608 19842 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:22.717069 19838 leader_election.cc:304] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:22.718645 19963 raft_consensus.cc:2749] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:22.718683 19838 leader_election.cc:304] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:22.725374 19963 raft_consensus.cc:2749] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:22.730580 19129 consensus_peers.cc:597] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.733999 19981 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34843
I20260709 12:14:22.734514 19981 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:22.735527 19981 heartbeater.cc:507] Master 127.9.11.126:34843 requested a full tablet report, sending...
I20260709 12:14:22.739565 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:22.740538 19705 consensus_queue.cc:1048] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:22.822261 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43888d54ff1d470a8d63fb846d1d9282" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:22.837685 18911 ts_manager.cc:194] Re-registered known tserver with Master: 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:22.889911 19842 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:22.891173 19842 tablet_bootstrap.cc:492] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:22.892022 19842 ts_tablet_manager.cc:1403] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.181s	user 0.038s	sys 0.016s
I20260709 12:14:22.912550 19842 raft_consensus.cc:359] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:22.913419 19842 raft_consensus.cc:740] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
W20260709 12:14:22.913610 19055 consensus_peers.cc:597] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.914180 19842 consensus_queue.cc:260] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:22.917291 19055 consensus_peers.cc:597] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.916777 19842 ts_tablet_manager.cc:1434] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:14:22.925227 19842 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
W20260709 12:14:22.933909 19055 consensus_peers.cc:597] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.940790 19055 consensus_peers.cc:597] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.942461 19203 consensus_peers.cc:597] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.944594 19203 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:22.948031 19203 consensus_peers.cc:597] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:22.948608 19838 leader_election.cc:304] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:22.949607 19963 raft_consensus.cc:2749] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:22.968760 19842 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:22.969936 19842 tablet_bootstrap.cc:492] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:22.970686 19842 ts_tablet_manager.cc:1403] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.000s
I20260709 12:14:22.971421 18911 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35490
I20260709 12:14:22.973922 19842 raft_consensus.cc:359] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.974750 19842 raft_consensus.cc:740] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:22.975598 19842 consensus_queue.cc:260] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:22.977689 19842 ts_tablet_manager.cc:1434] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:22.978905 19842 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:22.981348 19981 heartbeater.cc:499] Master 127.9.11.126:34843 was elected leader, sending a full tablet report...
I20260709 12:14:23.005417 19842 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.006479 19842 tablet_bootstrap.cc:492] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.007210 19842 ts_tablet_manager.cc:1403] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.015s
I20260709 12:14:23.009605 19842 raft_consensus.cc:359] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.010349 19842 raft_consensus.cc:740] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.011276 19842 consensus_queue.cc:260] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.013563 19842 ts_tablet_manager.cc:1434] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 12:14:23.014439 19842 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.030474 19842 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.031625 19842 tablet_bootstrap.cc:492] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.032490 19842 ts_tablet_manager.cc:1403] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 12:14:23.035404 19842 raft_consensus.cc:359] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.036190 19842 raft_consensus.cc:740] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.036980 19842 consensus_queue.cc:260] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.039477 19842 ts_tablet_manager.cc:1434] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:23.040441 19842 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.058410 19842 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.059677 19842 tablet_bootstrap.cc:492] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.060492 19842 ts_tablet_manager.cc:1403] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 12:14:23.063498 19842 raft_consensus.cc:359] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.064265 19842 raft_consensus.cc:740] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.065171 19842 consensus_queue.cc:260] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.067574 19842 ts_tablet_manager.cc:1434] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:23.068475 19842 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.077927 19890 raft_consensus.cc:3060] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:23.085104 19842 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.086341 19842 tablet_bootstrap.cc:492] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.087159 19842 ts_tablet_manager.cc:1403] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:14:23.090035 19842 raft_consensus.cc:359] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.090775 19842 raft_consensus.cc:740] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.091681 19842 consensus_queue.cc:260] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.094029 19842 ts_tablet_manager.cc:1434] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:14:23.095538 19842 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.106817 19890 raft_consensus.cc:3060] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:23.112332 19842 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.113579 19842 tablet_bootstrap.cc:492] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.114341 19842 ts_tablet_manager.cc:1403] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.002s
I20260709 12:14:23.117332 19842 raft_consensus.cc:359] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.118088 19842 raft_consensus.cc:740] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.118873 19842 consensus_queue.cc:260] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.121436 19842 ts_tablet_manager.cc:1434] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.006s
I20260709 12:14:23.124130 19842 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.153108 19890 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:23.194845 19842 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.196143 19842 tablet_bootstrap.cc:492] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.196931 19842 ts_tablet_manager.cc:1403] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.051s
I20260709 12:14:23.199820 19842 raft_consensus.cc:359] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.200645 19842 raft_consensus.cc:740] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.201532 19842 consensus_queue.cc:260] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.287652 19842 ts_tablet_manager.cc:1434] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.003s
I20260709 12:14:23.288936 19842 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.320822 19842 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.322084 19842 tablet_bootstrap.cc:492] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.322944 19842 ts_tablet_manager.cc:1403] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.005s
I20260709 12:14:23.328274 19842 raft_consensus.cc:359] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.329113 19842 raft_consensus.cc:740] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.330029 19842 consensus_queue.cc:260] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.332715 19842 ts_tablet_manager.cc:1434] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:14:23.333667 19842 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.357872 19842 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.358837 19842 tablet_bootstrap.cc:492] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.359548 19842 ts_tablet_manager.cc:1403] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.003s
I20260709 12:14:23.361784 19842 raft_consensus.cc:359] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.362385 19842 raft_consensus.cc:740] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.363042 19842 consensus_queue.cc:260] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.365224 19842 ts_tablet_manager.cc:1434] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:14:23.366796 19842 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.390293 19842 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.391531 19842 tablet_bootstrap.cc:492] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.392390 19842 ts_tablet_manager.cc:1403] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.010s
I20260709 12:14:23.395427 19842 raft_consensus.cc:359] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.396162 19842 raft_consensus.cc:740] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.396904 19842 consensus_queue.cc:260] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.399466 19842 ts_tablet_manager.cc:1434] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:14:23.400929 19842 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.423347 19842 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.424573 19842 tablet_bootstrap.cc:492] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.425473 19842 ts_tablet_manager.cc:1403] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.005s
I20260709 12:14:23.428108 19842 raft_consensus.cc:359] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.428699 19842 raft_consensus.cc:740] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.429510 19842 consensus_queue.cc:260] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.431955 19842 ts_tablet_manager.cc:1434] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:14:23.432881 19842 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.454656 19842 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.455894 19842 tablet_bootstrap.cc:492] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.456786 19842 ts_tablet_manager.cc:1403] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.005s
I20260709 12:14:23.460008 19842 raft_consensus.cc:359] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.460772 19842 raft_consensus.cc:740] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.461565 19842 consensus_queue.cc:260] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.464017 19842 ts_tablet_manager.cc:1434] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:23.465538 19842 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.485664 19842 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.486866 19842 tablet_bootstrap.cc:492] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.487751 19842 ts_tablet_manager.cc:1403] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 12:14:23.490785 19842 raft_consensus.cc:359] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.491617 19842 raft_consensus.cc:740] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.492439 19842 consensus_queue.cc:260] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.494831 19842 ts_tablet_manager.cc:1434] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:14:23.496501 19842 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.513453 19842 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.514434 19842 tablet_bootstrap.cc:492] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.515210 19842 ts_tablet_manager.cc:1403] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 12:14:23.517902 19842 raft_consensus.cc:359] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.518564 19842 raft_consensus.cc:740] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.519212 19842 consensus_queue.cc:260] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.521274 19842 ts_tablet_manager.cc:1434] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:14:23.522821 19842 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.538434 19842 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.539388 19842 tablet_bootstrap.cc:492] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.540007 19842 ts_tablet_manager.cc:1403] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.007s
I20260709 12:14:23.542142 19842 raft_consensus.cc:359] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.542811 19842 raft_consensus.cc:740] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.543493 19842 consensus_queue.cc:260] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.546152 19842 ts_tablet_manager.cc:1434] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:14:23.556602 19890 raft_consensus.cc:3060] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:23.572444 19842 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.602766 19890 raft_consensus.cc:3060] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:23.603780 19842 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.604940 19842 tablet_bootstrap.cc:492] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.605701 19842 ts_tablet_manager.cc:1403] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.010s
I20260709 12:14:23.609118 19842 raft_consensus.cc:359] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:23.609848 19842 raft_consensus.cc:740] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.610540 19842 consensus_queue.cc:260] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:23.612958 19842 ts_tablet_manager.cc:1434] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:23.613852 19842 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.648336 19842 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.649469 19842 tablet_bootstrap.cc:492] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.650282 19842 ts_tablet_manager.cc:1403] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.008s
I20260709 12:14:23.652760 19842 raft_consensus.cc:359] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.653479 19842 raft_consensus.cc:740] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.654068 19842 consensus_queue.cc:260] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.655905 19842 ts_tablet_manager.cc:1434] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:14:23.657241 19842 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.673986 19842 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.675154 19842 tablet_bootstrap.cc:492] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.675944 19842 ts_tablet_manager.cc:1403] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:14:23.678709 19842 raft_consensus.cc:359] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.679469 19842 raft_consensus.cc:740] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.680207 19842 consensus_queue.cc:260] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:23.682483 19842 ts_tablet_manager.cc:1434] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:14:23.683699 19842 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.706508 19842 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.714058 19842 tablet_bootstrap.cc:492] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.716403 19842 ts_tablet_manager.cc:1403] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.005s
I20260709 12:14:23.719460 19842 raft_consensus.cc:359] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.720320 19842 raft_consensus.cc:740] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.721199 19842 consensus_queue.cc:260] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.722944 19842 ts_tablet_manager.cc:1434] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:14:23.724663 19842 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.771317 19842 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.772830 19842 tablet_bootstrap.cc:492] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.773788 19842 ts_tablet_manager.cc:1403] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.000s
I20260709 12:14:23.776955 19842 raft_consensus.cc:359] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.777707 19842 raft_consensus.cc:740] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.778631 19842 consensus_queue.cc:260] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:23.781281 19842 ts_tablet_manager.cc:1434] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:14:23.782264 19842 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap starting.
I20260709 12:14:23.797881 19842 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:23.798784 19842 tablet_bootstrap.cc:492] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Bootstrap complete.
I20260709 12:14:23.799482 19842 ts_tablet_manager.cc:1403] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 12:14:23.801913 19842 raft_consensus.cc:359] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.802491 19842 raft_consensus.cc:740] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Initialized, Role: FOLLOWER
I20260709 12:14:23.803205 19842 consensus_queue.cc:260] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:23.805245 19842 ts_tablet_manager.cc:1434] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:14:23.861510 19891 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:24.309109 19885 raft_consensus.cc:3060] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:24.894392 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 49910.8us
I20260709 12:14:31.941388  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:31.999161 19234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:31.999914 19234 raft_consensus.cc:606] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.014344 19163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c"
dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:38990
I20260709 12:14:32.014961 19163 raft_consensus.cc:606] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.028577 19091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:32.029204 19091 raft_consensus.cc:606] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.031934 19234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:32.032459 19234 raft_consensus.cc:606] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.035128 19234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:32.035566 19234 raft_consensus.cc:606] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.038017 19234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:32.038599 19234 raft_consensus.cc:606] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.041236 19234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:32.041754 19234 raft_consensus.cc:606] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:32.043066  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:14:32.044740  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:32.045682  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:32.066185 20066 raft_consensus.cc:993] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.067027 20065 raft_consensus.cc:1081] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.069188 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:32.069877 19885 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.070346 19885 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.071221 20065 raft_consensus.cc:993] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.071795 20066 raft_consensus.cc:1081] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.073612 19891 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:32.074023 19891 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.074235 19891 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:32.081101 19885 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.082247 19891 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.085943 19885 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:32.086651 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:32.087561 19091 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.095182 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.095327 19891 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:32.096077 19238 raft_consensus.cc:3055] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:32.096513 19238 raft_consensus.cc:740] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:32.096681 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:32.097352 19090 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:32.097666 19238 consensus_queue.cc:260] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.099407 19238 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:32.099681 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.100322 19234 raft_consensus.cc:3055] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:32.096494 19091 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.102715 19234 raft_consensus.cc:740] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:32.108603 19838 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:32.109702 19234 consensus_queue.cc:260] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.110867 19238 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 3.
I20260709 12:14:32.111651 19234 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.115165 19838 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:32.114794 20073 raft_consensus.cc:2804] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:32.120779 19090 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 3.
I20260709 12:14:32.129005 19234 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.147950 20073 raft_consensus.cc:697] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.149250 20065 raft_consensus.cc:993] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.150923 20066 raft_consensus.cc:1081] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.148099 20074 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:32.155365 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:32.156282 19885 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.156769 19885 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.155880 20073 consensus_queue.cc:237] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.158203 20074 raft_consensus.cc:697] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.159338 20074 consensus_queue.cc:237] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.166265 19885 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.169487 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:32.170114 19163 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.175925 19885 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:32.177222 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.177986 19234 raft_consensus.cc:3055] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:32.178345 19234 raft_consensus.cc:740] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:32.179323 19234 consensus_queue.cc:260] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.180852 19234 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.182430 18908 catalog_manager.cc:5697] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:32.187821 19163 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.190440 19234 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.191960 19837 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:32.193060 20074 raft_consensus.cc:2804] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:32.193859 20074 raft_consensus.cc:697] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.194888 20074 consensus_queue.cc:237] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.202373 20068 raft_consensus.cc:993] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.203125 20068 raft_consensus.cc:1081] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.205369 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54064
I20260709 12:14:32.206012 19885 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.206395 19885 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.215426 19885 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.218219 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.218855 19234 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.219655 19885 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:32.222268 19090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:32.222829 19090 raft_consensus.cc:3055] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:32.223163 19090 raft_consensus.cc:740] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:32.223968 19090 consensus_queue.cc:260] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.225273 19090 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.227358 19234 raft_consensus.cc:2468] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.228789 19838 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:32.231906 20074 raft_consensus.cc:2804] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:32.232460 20074 raft_consensus.cc:697] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.233515 20074 consensus_queue.cc:237] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.233328 18908 catalog_manager.cc:5697] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 2 to 3, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:32.235600 18908 catalog_manager.cc:5697] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:32.245271 19090 raft_consensus.cc:2468] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.268867 18908 catalog_manager.cc:5697] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:32.307155 20065 raft_consensus.cc:993] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.307914 20076 raft_consensus.cc:1081] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.309787 19890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:32.310782 19890 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.311198 19890 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.314018 20050 raft_consensus.cc:993] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.314651 20050 raft_consensus.cc:1081] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.317130 19889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54074
I20260709 12:14:32.317667 19889 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.317998 19889 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.321079 19890 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.323787 19890 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:32.324455 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.325169 19234 raft_consensus.cc:3055] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:32.325475 19234 raft_consensus.cc:740] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:32.325701 19091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:32.326390 19091 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.326342 19234 consensus_queue.cc:260] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.327951 19234 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.336236 19889 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.337446 19091 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.338692 19838 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:32.338954 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:32.339681 20074 raft_consensus.cc:2804] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:32.339771 19163 raft_consensus.cc:3055] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:32.340235 19163 raft_consensus.cc:740] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:32.340271 20074 raft_consensus.cc:697] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.341250 20074 consensus_queue.cc:237] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:32.341063 19163 consensus_queue.cc:260] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.341634 19234 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.342327 19889 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:32.342921 19163 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.345078 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.345815 19234 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.354940 19163 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.356630 19234 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.357278 18908 catalog_manager.cc:5697] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:32.358124 19838 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:32.358486 20076 raft_consensus.cc:993] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.359500 20065 raft_consensus.cc:1081] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:32.359696 20074 raft_consensus.cc:2804] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:32.360291 20074 raft_consensus.cc:697] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.361323 20074 consensus_queue.cc:237] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.363826 19889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:32.364449 19889 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:32.364823 19889 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.379336 19889 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.381610 19889 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:32.382640 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:32.382452 18908 catalog_manager.cc:5697] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:32.383412 19234 raft_consensus.cc:3055] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:32.383507 19164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:32.383826 19234 raft_consensus.cc:740] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:32.384243 19164 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.384846 19234 consensus_queue.cc:260] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.386418 19234 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:32.400177 19164 raft_consensus.cc:2468] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.400379 19234 raft_consensus.cc:2468] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:32.401904 19837 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:32.402670 20074 raft_consensus.cc:2804] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:32.403256 20074 raft_consensus.cc:697] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:32.404106 20074 consensus_queue.cc:237] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:32.420601 18908 catalog_manager.cc:5697] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:32.588958 19239 raft_consensus.cc:1275] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.590354 20074 consensus_queue.cc:1048] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:32.603825 19164 raft_consensus.cc:1275] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.605388 20073 consensus_queue.cc:1048] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:32.625065 19091 raft_consensus.cc:1275] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:32.626976 20073 consensus_queue.cc:1048] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:32.658584 19234 raft_consensus.cc:1275] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.660629 19239 raft_consensus.cc:1275] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:32.662344 20074 consensus_queue.cc:1048] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:32.664188 20074 consensus_queue.cc:1048] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:14:32.818197 19129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 100282us
W20260709 12:14:32.876437 19055 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 48914.8us
W20260709 12:14:32.892122 19838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 51842.7us
I20260709 12:14:32.900004 19089 raft_consensus.cc:1275] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.901502 19234 raft_consensus.cc:1275] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.902201 19088 raft_consensus.cc:1275] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.913456 19234 raft_consensus.cc:1275] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.916565 19234 raft_consensus.cc:1275] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:14:32.923503 19130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:34717, user_credentials={real_user=slave}} blocked reactor thread for 46884.3us
I20260709 12:14:32.926964 20109 consensus_queue.cc:1048] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:32.927893 20113 consensus_queue.cc:1048] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:32.931461 20103 consensus_queue.cc:1048] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:32.933698 20073 consensus_queue.cc:1048] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:32.936498 19234 raft_consensus.cc:1275] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.930106 20108 consensus_queue.cc:1048] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:32.938035 19089 raft_consensus.cc:1275] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.940742 20113 consensus_queue.cc:1048] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:32.952939 19160 raft_consensus.cc:1275] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.963902 20113 consensus_queue.cc:1048] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:32.981056 19161 raft_consensus.cc:1275] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:32.982924 20073 consensus_queue.cc:1048] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:32.997205 20103 consensus_queue.cc:1048] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:33.095898  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:33.142390 19160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1"
dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:38990
I20260709 12:14:33.143054 19160 raft_consensus.cc:606] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:33.145813 19086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:33.146441 19086 raft_consensus.cc:606] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Received request to transfer leadership to TS 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:33.149197 19086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "6cab1cfd695145f992765d39dbe662c7"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:33.149701 19086 raft_consensus.cc:606] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Received request to transfer leadership to TS 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:33.152477 19086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:33.153040 19086 raft_consensus.cc:606] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Received request to transfer leadership to TS 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:33.155689 19086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edf86dd5411044549e507998d7b14952"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:33.156153 19086 raft_consensus.cc:606] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:33.158519 19086 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:33.159157 19086 raft_consensus.cc:606] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:33.160471  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:33.162657  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:33.163441  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:33.238554 20050 raft_consensus.cc:993] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:33.239315 20116 raft_consensus.cc:1081] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:33.241001 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54074
I20260709 12:14:33.241564 19885 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.241884 19885 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.250417 19885 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:33.252658 19885 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:33.253718 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:33.254580 19234 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.255664 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:33.256386 19160 raft_consensus.cc:3055] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:33.256687 19160 raft_consensus.cc:740] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:33.257555 19160 consensus_queue.cc:260] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:33.258961 19160 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.263549 19234 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:33.265487 19838 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:33.266582 20075 raft_consensus.cc:2804] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:33.267165 20075 raft_consensus.cc:697] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:33.268187 20075 consensus_queue.cc:237] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:33.271183 19160 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:33.285032 18911 catalog_manager.cc:5697] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.369800 20106 raft_consensus.cc:993] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:33.370554 20107 raft_consensus.cc:1081] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:33.372409 19885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54064
I20260709 12:14:33.373087 19885 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.373448 19885 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:33.379884 19885 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.381596 19885 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:33.382849 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:33.382951 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:33.383611 19234 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:33.383718 19086 raft_consensus.cc:3055] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:33.384199 19086 raft_consensus.cc:740] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:33.385107 19086 consensus_queue.cc:260] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.386441 19086 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:33.390226 19234 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 3.
I20260709 12:14:33.391975 19838 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:33.392997 20075 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:33.393729 20075 raft_consensus.cc:697] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:33.396847 20075 consensus_queue.cc:237] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.397882 19086 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 3.
I20260709 12:14:33.410419 18911 catalog_manager.cc:5697] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 2 to 3, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 3 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:33.432471 20047 raft_consensus.cc:993] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 6cab1cfd695145f992765d39dbe662c7 to start an election
I20260709 12:14:33.433095 20047 raft_consensus.cc:1081] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Signalling peer 6cab1cfd695145f992765d39dbe662c7 to start an election
I20260709 12:14:33.435062 19160 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961"
dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
 from {username='slave'} at 127.0.0.1:36934
I20260709 12:14:33.435655 19160 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.436002 19160 raft_consensus.cc:3060] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.444571 20047 raft_consensus.cc:993] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 2c0b3983ae6c48108f6f3632116cf5a8 to start an election
I20260709 12:14:33.445227 20047 raft_consensus.cc:1081] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Signalling peer 2c0b3983ae6c48108f6f3632116cf5a8 to start an election
I20260709 12:14:33.447084 19234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
 from {username='slave'} at 127.0.0.1:45324
I20260709 12:14:33.447649 19234 raft_consensus.cc:493] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.447999 19234 raft_consensus.cc:3060] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.454097 19160 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.456918 19160 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:33.458060 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:33.458528 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a1299cc26f84cbaa6278121975f7961" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:33.458861 19237 raft_consensus.cc:3060] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.459282 19086 raft_consensus.cc:3055] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:33.459651 19086 raft_consensus.cc:740] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:33.460573 19086 consensus_queue.cc:260] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.460632 19234 raft_consensus.cc:515] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.461892 19086 raft_consensus.cc:3060] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.464687 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:33.465448 19089 raft_consensus.cc:3055] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:33.465762 19089 raft_consensus.cc:740] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:33.466626 19089 consensus_queue.cc:260] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.467868 19089 raft_consensus.cc:3060] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.468791 19237 raft_consensus.cc:2468] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:33.471901 19130 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:33.473156 20116 raft_consensus.cc:2804] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:33.473681 20116 raft_consensus.cc:697] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:33.474707 20116 consensus_queue.cc:237] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.476773 19234 leader_election.cc:290] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:33.479393 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9af570d255149c5aedd8856d3698e12" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:33.481256 19160 raft_consensus.cc:3060] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.495142 19089 raft_consensus.cc:2468] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:33.497068 19204 leader_election.cc:304] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:33.498230 20112 raft_consensus.cc:2804] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:33.498822 20112 raft_consensus.cc:697] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:33.499924 19160 raft_consensus.cc:2468] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:33.500610 20112 consensus_queue.cc:237] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.501559 19086 raft_consensus.cc:2468] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:33.509419 18911 catalog_manager.cc:5697] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 1 to 2, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:33.515328 18910 catalog_manager.cc:5697] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 1 to 2, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 2 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.557283 20106 raft_consensus.cc:993] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:33.557837 20107 raft_consensus.cc:1081] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:33.559578 19889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edf86dd5411044549e507998d7b14952"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54064
I20260709 12:14:33.560170 19889 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.560498 19889 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.569300 19889 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:33.571244 19889 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:33.572885 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:33.573107 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:33.573762 19160 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.573823 19086 raft_consensus.cc:3055] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:33.574316 19086 raft_consensus.cc:740] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:33.575397 19086 consensus_queue.cc:260] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:33.576946 19086 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.583544 19160 raft_consensus.cc:2468] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:33.585835 19837 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:33.586656 20075 raft_consensus.cc:2804] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:33.587244 20075 raft_consensus.cc:697] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:33.588251 20075 consensus_queue.cc:237] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:33.591603 19086 raft_consensus.cc:2468] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:33.602129 18910 catalog_manager.cc:5697] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:33.621677 20106 raft_consensus.cc:993] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 2c0b3983ae6c48108f6f3632116cf5a8 to start an election
I20260709 12:14:33.622260 20107 raft_consensus.cc:1081] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Signalling peer 2c0b3983ae6c48108f6f3632116cf5a8 to start an election
I20260709 12:14:33.624099 19234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
 from {username='slave'} at 127.0.0.1:45324
I20260709 12:14:33.624737 19234 raft_consensus.cc:493] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.625137 19234 raft_consensus.cc:3060] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.633210 19234 raft_consensus.cc:515] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.635012 19234 leader_election.cc:290] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:33.635846 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:33.636191 19161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:33.636683 19086 raft_consensus.cc:3055] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:33.636956 19161 raft_consensus.cc:3060] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.637120 19086 raft_consensus.cc:740] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:33.638021 19086 consensus_queue.cc:260] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.639292 19086 raft_consensus.cc:3060] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:33.644956 19161 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:33.646232 19086 raft_consensus.cc:2468] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:33.646531 19203 leader_election.cc:304] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:33.647488 20066 raft_consensus.cc:2804] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:33.647948 20066 raft_consensus.cc:697] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:33.648815 20066 consensus_queue.cc:237] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:33.665490 18910 catalog_manager.cc:5697] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 1 to 2, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 2 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:33.671938 19234 raft_consensus.cc:1275] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:33.677562 20075 consensus_queue.cc:1048] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:33.695374 19161 raft_consensus.cc:1275] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:33.697500 20103 consensus_queue.cc:1048] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:33.828138 19237 raft_consensus.cc:1275] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:33.829545 20103 consensus_queue.cc:1048] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:33.840603 19086 raft_consensus.cc:1275] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:33.848640 20075 consensus_queue.cc:1048] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:33.893000 19086 raft_consensus.cc:1275] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:33.894398 20116 consensus_queue.cc:1048] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:33.904994 19234 raft_consensus.cc:1275] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:33.907478 20116 consensus_queue.cc:1048] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:33.973106 19160 raft_consensus.cc:1275] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:33.975383 20066 consensus_queue.cc:1048] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:33.997207 19086 raft_consensus.cc:1275] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.002259 20066 consensus_queue.cc:1048] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:34.126035 19086 raft_consensus.cc:1275] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.127490 20149 consensus_queue.cc:1048] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.140625 19160 raft_consensus.cc:1275] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.143231 20066 consensus_queue.cc:1048] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.150023 19161 raft_consensus.cc:1275] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.151881 20075 consensus_queue.cc:1048] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:34.184314 19086 raft_consensus.cc:1275] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.186348 20103 consensus_queue.cc:1048] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.214682  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:34.259949 19089 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff"
dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
mode: GRACEFUL
new_leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
 from {username='slave'} at 127.0.0.1:55014
I20260709 12:14:34.260594 19089 raft_consensus.cc:606] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Received request to transfer leadership to TS 2c0b3983ae6c48108f6f3632116cf5a8
I20260709 12:14:34.263595 19237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:34.264112 19237 raft_consensus.cc:606] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:34.267117 19237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:34.267671 19237 raft_consensus.cc:606] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:34.272490 19160 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4"
dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:38990
I20260709 12:14:34.273128 19160 raft_consensus.cc:606] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:34.277613  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:14:34.280385  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:34.281239  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:34.297408 20147 raft_consensus.cc:993] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:34.298075 20050 raft_consensus.cc:1081] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:34.300055 19890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54074
I20260709 12:14:34.300741 19890 raft_consensus.cc:493] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:34.301157 19890 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.311448 19890 raft_consensus.cc:515] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:34.314952 19890 leader_election.cc:290] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:34.315521 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:34.316224 19089 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.324213 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:34.325043 19160 raft_consensus.cc:3055] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:34.325335 19160 raft_consensus.cc:740] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:34.326169 19160 consensus_queue.cc:260] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:34.327680 19160 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.334570 19089 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:34.335984 20107 raft_consensus.cc:993] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: : Instructing follower 2c0b3983ae6c48108f6f3632116cf5a8 to start an election
I20260709 12:14:34.336550 20107 raft_consensus.cc:1081] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Signalling peer 2c0b3983ae6c48108f6f3632116cf5a8 to start an election
I20260709 12:14:34.338792 19239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
 from {username='slave'} at 127.0.0.1:45324
I20260709 12:14:34.339422 19239 raft_consensus.cc:493] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:34.339768 19239 raft_consensus.cc:3060] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.343775 19160 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:34.345000 19837 leader_election.cc:304] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:34.345813 20142 raft_consensus.cc:2804] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:34.346359 20142 raft_consensus.cc:697] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:34.347325 20142 consensus_queue.cc:237] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:34.348912 19239 raft_consensus.cc:515] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:34.351109 19239 leader_election.cc:290] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Requested vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:34.352319 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:34.353045 19089 raft_consensus.cc:3055] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:34.353274 19089 raft_consensus.cc:740] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:34.354010 19089 consensus_queue.cc:260] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:34.355165 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ccbbf4c1622458faa667dc368b16eff" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:34.355280 19089 raft_consensus.cc:3060] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.359634 19160 raft_consensus.cc:3060] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.368384 19089 raft_consensus.cc:2468] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:34.371121 19160 raft_consensus.cc:2468] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:34.371308 19204 leader_election.cc:304] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:34.369562 18911 catalog_manager.cc:5697] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:34.375152 20149 raft_consensus.cc:2804] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:34.375699 20149 raft_consensus.cc:697] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:34.376813 20149 consensus_queue.cc:237] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:34.382217 20149 raft_consensus.cc:993] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:34.382901 20149 raft_consensus.cc:1081] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:34.384833 19890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:34.387326 19890 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:34.387748 19890 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.399495 19890 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:34.395231 18910 catalog_manager.cc:5697] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 1 to 2, leader changed from 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 2 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:34.402344 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:34.403008 19234 raft_consensus.cc:3055] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:34.403322 19234 raft_consensus.cc:740] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:34.404155 19234 consensus_queue.cc:260] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:34.405035 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:34.404798 19890 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:34.405642 19086 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.407090 19234 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.415673 19234 raft_consensus.cc:2468] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:34.417105 19838 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1; no voters: 
I20260709 12:14:34.418028 20142 raft_consensus.cc:2804] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:34.418598 20142 raft_consensus.cc:697] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:34.418704 19086 raft_consensus.cc:2468] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:34.419708 20142 consensus_queue.cc:237] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:34.430563 18910 catalog_manager.cc:5697] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:34.448132 20150 raft_consensus.cc:993] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:34.448797 20150 raft_consensus.cc:1081] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:34.450416 19890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:34.451046 19890 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:34.451327 19890 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.457065 19890 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:34.458655 19890 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:34.459936 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:34.460279 19234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:34.460651 19086 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.460973 19234 raft_consensus.cc:3055] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:34.461329 19234 raft_consensus.cc:740] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:34.462265 19234 consensus_queue.cc:260] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:34.463616 19234 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:34.471774 19234 raft_consensus.cc:2468] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:34.471819 19086 raft_consensus.cc:2468] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:34.473104 19838 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:34.473873 20142 raft_consensus.cc:2804] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:34.474385 20142 raft_consensus.cc:697] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:34.475219 20142 consensus_queue.cc:237] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:34.489317 18910 catalog_manager.cc:5697] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:34.793349 19161 raft_consensus.cc:1275] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.794893 20159 consensus_queue.cc:1048] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.826221 19086 raft_consensus.cc:1275] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.827940 20066 consensus_queue.cc:1048] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.837934 19089 raft_consensus.cc:1275] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.839592 20142 consensus_queue.cc:1048] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.851951 19161 raft_consensus.cc:1275] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.862498 20073 consensus_queue.cc:1048] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.880432 19234 raft_consensus.cc:1275] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.883384 20075 consensus_queue.cc:1048] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.905283 19086 raft_consensus.cc:1275] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.907337 20142 consensus_queue.cc:1048] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.926316 19086 raft_consensus.cc:1275] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.927590 20142 consensus_queue.cc:1048] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:34.954461 19234 raft_consensus.cc:1275] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:34.956216 20142 consensus_queue.cc:1048] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:14:35.196715 19129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 65093.6us
I20260709 12:14:35.341415  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:35.405676 19237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870"
dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
mode: GRACEFUL
new_leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:52010
I20260709 12:14:35.406626 19237 raft_consensus.cc:606] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Received request to transfer leadership to TS 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:35.409597  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:14:35.412655  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:35.413437  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:35.921180 20159 raft_consensus.cc:993] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: : Instructing follower 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:35.921809 20178 raft_consensus.cc:1081] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Signalling peer 3fcac1aec8df4c599b4c67550ce347e1 to start an election
I20260709 12:14:35.923628 19889 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870"
dest_uuid: "3fcac1aec8df4c599b4c67550ce347e1"
 from {username='slave'} at 127.0.0.1:54070
I20260709 12:14:35.924259 19889 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:35.924705 19889 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:35.935668 19889 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:35.937573 19889 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:35.938838 19163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:35.938809 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "3fcac1aec8df4c599b4c67550ce347e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:35.939899 19163 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:35.939971 19237 raft_consensus.cc:3055] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:35.940411 19237 raft_consensus.cc:740] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:35.941289 19237 consensus_queue.cc:260] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:35.942615 19237 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:35.947808 19163 raft_consensus.cc:2468] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:35.949415 19837 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:35.950425 20073 raft_consensus.cc:2804] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:35.950543 19237 raft_consensus.cc:2468] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fcac1aec8df4c599b4c67550ce347e1 in term 2.
I20260709 12:14:35.951074 20073 raft_consensus.cc:697] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Becoming Leader. State: Replica: 3fcac1aec8df4c599b4c67550ce347e1, State: Running, Role: LEADER
I20260709 12:14:35.952346 20073 consensus_queue.cc:237] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:35.968181 18911 catalog_manager.cc:5697] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 reported cstate change: term changed from 1 to 2, leader changed from 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68) to 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "3fcac1aec8df4c599b4c67550ce347e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:36.431727 19160 raft_consensus.cc:1275] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:36.432999 20073 consensus_queue.cc:1048] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:36.453755  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:36.457381 19234 raft_consensus.cc:1275] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Refusing update from remote peer 3fcac1aec8df4c599b4c67550ce347e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:36.459515 20142 consensus_queue.cc:1048] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:36.514334  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:14:36.516140  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:36.516831  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:36.553663  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:36.595686  9261 tablet_server.cc:179] TabletServer@127.9.11.65:43327 shutting down...
W20260709 12:14:36.619546 19129 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 190 similar messages]
W20260709 12:14:36.630396 19129 consensus_peers.cc:597] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.653198  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:36.655390  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.657215  9261 raft_consensus.cc:2243] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.658077  9261 raft_consensus.cc:2272] T caf9e5cd30214f99a71d1f25f34dae84 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.669070  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.670840  9261 raft_consensus.cc:2243] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.671959  9261 raft_consensus.cc:2272] T 62d133466fbe4df8a6e09156cec6acf4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.672811 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.685009 19055 consensus_peers.cc:597] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.691524  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.693061  9261 raft_consensus.cc:2243] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.693872  9261 raft_consensus.cc:2272] T 718f20f68825449c836f67bd5a4bae55 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.697160 19129 consensus_peers.cc:597] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.703745 19129 consensus_peers.cc:597] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.706051 19203 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.709661 19055 consensus_peers.cc:597] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.710974  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.712553  9261 raft_consensus.cc:2243] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.714222  9261 raft_consensus.cc:2272] T 5373fb94420c4369bab88209ddbd5324 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.718747  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.720057  9261 raft_consensus.cc:2243] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.720738  9261 raft_consensus.cc:2272] T 81a0d1ae8dd1431d9f241ba27f35f18f P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.721448 19129 consensus_peers.cc:597] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.725188 19129 consensus_peers.cc:597] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.725271  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.726368  9261 raft_consensus.cc:2243] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.727502  9261 raft_consensus.cc:2272] T c0fdf1bfb7c54f7da0aa45218c041b08 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.730537  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.731493  9261 raft_consensus.cc:2243] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.732712  9261 raft_consensus.cc:2272] T bee0ab48d58247cf8e32af4e5a43e66a P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.735527  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.736392  9261 raft_consensus.cc:2243] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.736959  9261 raft_consensus.cc:2272] T 54df033d2cbe422390e0fa4bf287ee14 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.741433  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.742551  9261 raft_consensus.cc:2243] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.743985  9261 raft_consensus.cc:2272] T d99ec37464bd4047a9485e16e72301e9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.747738  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.748577  9261 raft_consensus.cc:2243] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.749205  9261 raft_consensus.cc:2272] T 5ecdce0779954a328062dfca1a0364e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.752508  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.753435  9261 raft_consensus.cc:2243] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.753999  9261 raft_consensus.cc:2272] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.757103  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.757912  9261 raft_consensus.cc:2243] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.758507  9261 raft_consensus.cc:2272] T d311e304cfc84db5a4dfc30459f4d72e P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.761579  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.762478  9261 raft_consensus.cc:2243] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.763644  9261 raft_consensus.cc:2272] T 3cfc76678bd1474fb6f9826360c9e870 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.767158  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.768198  9261 raft_consensus.cc:2243] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.768925  9261 raft_consensus.cc:2272] T c394ca853d394fdbafaa3a25b2633648 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.772729  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.773458  9261 raft_consensus.cc:2243] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.774701  9261 raft_consensus.cc:2272] T c890b118e3b14e4291cb99b7c9f31ae2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.778003  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.778841  9261 raft_consensus.cc:2243] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.780349  9261 raft_consensus.cc:2272] T fbccd6116cc5452ebe3b6f446963bcd2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.783766  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.784811  9261 raft_consensus.cc:2243] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.785382  9261 raft_consensus.cc:2272] T 2ff6615d74da465f9105797edb9c3698 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.788283  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.789202  9261 raft_consensus.cc:2243] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.789865  9261 raft_consensus.cc:2272] T 20ddebbef4404377b6b25cc37866b9e2 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.794152  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.795229  9261 raft_consensus.cc:2243] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.796638  9261 raft_consensus.cc:2272] T df92e505d2454dea8eabdad38c8f36e4 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.797750 19203 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.799947  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.801023  9261 raft_consensus.cc:2243] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.801517  9261 raft_consensus.cc:2272] T 2b5f4455e6964b99b522e009ca88b6d8 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.804517  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.805452  9261 raft_consensus.cc:2243] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.806854  9261 raft_consensus.cc:2272] T 9ec43e6601ae4d8cab4e09c7e77632ae P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.810114  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.810920  9261 raft_consensus.cc:2243] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.811556  9261 raft_consensus.cc:2272] T 43f821d9236644bcb6676552b6ee4019 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.815429  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.816483  9261 raft_consensus.cc:2243] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.817126  9261 raft_consensus.cc:2272] T 20a717f41e3045c8b600ca072c4b1efd P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.821208  9261 tablet_replica.cc:333] stopping tablet replica
W20260709 12:14:36.821911 19055 consensus_peers.cc:597] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.822211  9261 raft_consensus.cc:2243] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.823040  9261 raft_consensus.cc:2272] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.823566 19055 consensus_peers.cc:597] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.825882  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.826682  9261 raft_consensus.cc:2243] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.827380  9261 raft_consensus.cc:2272] T 43888d54ff1d470a8d63fb846d1d9282 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.830317  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.830931  9261 raft_consensus.cc:2243] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.831414  9261 raft_consensus.cc:2272] T 6079f38db31043d2bc846667f08d7d1b P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.834131  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.834925  9261 raft_consensus.cc:2243] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.835486  9261 raft_consensus.cc:2272] T d409ffd2084d48bf8886755a2146b5e0 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.838603  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.839537  9261 raft_consensus.cc:2243] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.840171  9261 raft_consensus.cc:2272] T 9c4c556a691543529aee3c07a4015d8d P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.840972 19203 consensus_peers.cc:597] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.843278  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.844100  9261 raft_consensus.cc:2243] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.844839  9261 raft_consensus.cc:2272] T 3d1280ae7e994dc59cf16f8ddef18b72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.848387  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.849432  9261 raft_consensus.cc:2243] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.850124  9261 raft_consensus.cc:2272] T 428f9c2176004b6aa007c6a7af377fcb P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.852428 19055 consensus_peers.cc:597] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.853871  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.854954  9261 raft_consensus.cc:2243] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.856401  9261 raft_consensus.cc:2272] T 5d792b44d6c54e1490e101d60dbebb82 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.859359  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.860072  9261 raft_consensus.cc:2243] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.861102  9261 raft_consensus.cc:2272] T edf86dd5411044549e507998d7b14952 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.863719  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.864403  9261 raft_consensus.cc:2243] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.865489  9261 raft_consensus.cc:2272] T 169518fbf85c4378af0ff5af5df6dff1 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.868263  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.868916  9261 raft_consensus.cc:2243] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.869472  9261 raft_consensus.cc:2272] T 64a4de1a07654315ae7678c1f9e8fcca P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.871980  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.872629  9261 raft_consensus.cc:2243] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.873095  9261 raft_consensus.cc:2272] T a377e1ad0954438589b509e2491c8a72 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.876060  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.876952  9261 raft_consensus.cc:2243] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.877583  9261 raft_consensus.cc:2272] T 04add48cb044473fbc449788540f24c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.881049  9261 tablet_replica.cc:333] stopping tablet replica
W20260709 12:14:36.881230 19055 consensus_peers.cc:597] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.881958  9261 raft_consensus.cc:2243] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.882748  9261 raft_consensus.cc:2272] T 0eed4a045f6747428301e56cf20644b7 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.886355  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.887089  9261 raft_consensus.cc:2243] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.887562  9261 raft_consensus.cc:2272] T 55dd90070791474ea046bd86b13e3f75 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.890575  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.891275  9261 raft_consensus.cc:2243] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.891718  9261 raft_consensus.cc:2272] T 7726c731410a492188ef1b403a54f3bc P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.894434  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.895295  9261 raft_consensus.cc:2243] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.896615  9261 raft_consensus.cc:2272] T e913e5c22bd646e1944cf0bac9dae4c9 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.900281  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.901095  9261 raft_consensus.cc:2243] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.902123  9261 raft_consensus.cc:2272] T ee5a381266c643c9a8b1ea339f87b328 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.904867  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.905505  9261 raft_consensus.cc:2243] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.905957  9261 raft_consensus.cc:2272] T c3afa3fafa244643a6da11e31e841d34 P 3fcac1aec8df4c599b4c67550ce347e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.908357  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.908941  9261 raft_consensus.cc:2243] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:36.909996  9261 raft_consensus.cc:2272] T 9061504a72094d2abbe26a00b4998e9c P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:36.912638  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.913482  9261 raft_consensus.cc:2243] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.914158  9261 raft_consensus.cc:2272] T e8e8cd17ef794dd18367b9052aae4c29 P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.916950 19129 consensus_peers.cc:597] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.917383  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:36.918226  9261 raft_consensus.cc:2243] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:36.918789  9261 raft_consensus.cc:2272] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 3fcac1aec8df4c599b4c67550ce347e1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:36.950770 19203 consensus_peers.cc:597] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.969887 19129 consensus_peers.cc:597] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:36.992291 19129 consensus_peers.cc:597] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.005102 19203 consensus_peers.cc:597] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.012948 19129 consensus_peers.cc:597] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.687512 19203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.020s
W20260709 12:14:37.693711 19203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.006s	sys 0.021s
I20260709 12:14:37.705250 20178 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:37.706704 20178 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:37.712007 20178 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:37.768226 19203 consensus_peers.cc:597] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.769090 19203 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:37.769695 19203 consensus_peers.cc:597] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.770244 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.770725 19203 consensus_peers.cc:597] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.771682 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.782975 19055 consensus_peers.cc:597] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:37.841727  9261 tablet_server.cc:196] TabletServer@127.9.11.65:43327 shutdown complete.
W20260709 12:14:37.858090 19055 consensus_peers.cc:597] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:37.879685 20212 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:37.880402 20212 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:37.883773 20212 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:37.886551 20212 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:37.887233 20212 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:37.889302 19055 leader_election.cc:336] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:37.892553 20212 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:37.898310 19055 leader_election.cc:336] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:37.915134 20227 raft_consensus.cc:493] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:37.916071 20227 raft_consensus.cc:515] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:37.918838 20227 leader_election.cc:290] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:37.923041 19129 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 101728us
W20260709 12:14:37.936183 19129 consensus_peers.cc:597] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.937902 19129 consensus_peers.cc:597] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:37.941452 20227 raft_consensus.cc:493] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2c0b3983ae6c48108f6f3632116cf5a8)
I20260709 12:14:37.942155 20227 raft_consensus.cc:515] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:37.944708 20227 leader_election.cc:290] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:37.951116 20226 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
W20260709 12:14:37.953786 19129 consensus_peers.cc:597] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.955464 19129 leader_election.cc:336] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:37.953279 20226 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:37.958593 19129 leader_election.cc:336] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:37.965951 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:37.966526 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:37.962410 20226 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:37.970654 20159 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:37.971387 20159 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:37.972342 19056 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:37.973526 20212 raft_consensus.cc:2749] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:37.973657 20159 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:37.975032 19056 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:37.983146 20207 raft_consensus.cc:2749] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:37.979224 19129 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:37.984751 19089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:37.985394 20159 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:37.986020 20159 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:37.988639 20159 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:37.995632 20226 raft_consensus.cc:493] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2c0b3983ae6c48108f6f3632116cf5a8)
I20260709 12:14:37.996394 20226 raft_consensus.cc:515] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:37.999221 20226 leader_election.cc:290] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:38.015388 20205 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.016080 20205 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.016649 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.018422 20205 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.022019 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.024891 20207 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.025563 20207 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.027930 20207 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:38.030907 19203 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:38.031528 19203 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.031940 19203 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:38.032383 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.032933 19130 leader_election.cc:304] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:38.033080 19239 raft_consensus.cc:2468] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:38.034379 19056 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.035367 20159 raft_consensus.cc:2749] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:38.035847 19055 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.035998 20207 raft_consensus.cc:2804] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.036458 20207 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.036803 20207 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.035338 20216 raft_consensus.cc:2749] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:38.044752 19203 leader_election.cc:336] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.045358 19203 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:38.046471 20159 raft_consensus.cc:2749] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.051980 19160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:38.052726 19160 raft_consensus.cc:2468] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:38.052790 19158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:38.053321 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.048987 20207 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.056577 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c890b118e3b14e4291cb99b7c9f31ae2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:38.057309 19239 raft_consensus.cc:3060] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.067018 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7726c731410a492188ef1b403a54f3bc" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.057075 20207 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.068409 19239 raft_consensus.cc:2468] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 3.
I20260709 12:14:38.069870 19056 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.070842 20207 raft_consensus.cc:2804] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.071650 20207 raft_consensus.cc:697] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:38.072820 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
W20260709 12:14:38.072908 19055 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.073670 19236 raft_consensus.cc:2468] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:38.072666 20207 consensus_queue.cc:237] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.079370 19130 leader_election.cc:304] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.080230 20227 raft_consensus.cc:2749] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:38.080711 19130 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.081630 20227 raft_consensus.cc:2804] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.081986 20227 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.082194 19203 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.082362 20227 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.083285 20204 raft_consensus.cc:2804] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.083688 20204 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.084069 20204 raft_consensus.cc:3060] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.086690 19203 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:38.087630 20159 raft_consensus.cc:2749] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.089458 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3e80b40cf94fca9db5cc0ea02e4c48" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.092267 19130 leader_election.cc:304] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:38.093111 20221 raft_consensus.cc:2749] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.097678 20204 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.101167 20227 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.111207 19156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:38.111807 20178 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.112572 20178 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.112874 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cfc76678bd1474fb6f9826360c9e870" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
W20260709 12:14:38.113586 19129 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.113868 19236 raft_consensus.cc:2393] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 3: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
I20260709 12:14:38.114428 20227 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.114414 20219 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.115317 20219 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.116598 19130 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.117581 20221 raft_consensus.cc:2749] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.117903 20219 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.117547 20178 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.121667 20219 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.122362 20219 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:38.127463 19203 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.130867 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.132251 20222 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.132478 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.133163 19086 raft_consensus.cc:2468] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:38.132944 20222 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.135182 20219 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.130649 20221 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.136502 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.137689 19236 raft_consensus.cc:2468] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:38.137498 20221 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:38.135921 19203 leader_election.cc:336] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.140093 19056 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.143297 20209 raft_consensus.cc:2804] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.143795 20209 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.144184 20209 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.146881 19203 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.147372 19156 raft_consensus.cc:2393] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 3: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
W20260709 12:14:38.147871 19129 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.149183 20221 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.150308 20222 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.156644 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.157167 20219 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.157678 19236 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:38.157878 20219 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.158648 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.160487 19204 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.160792 20219 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.161993 19204 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:38.162263 19130 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.163657 20178 raft_consensus.cc:2804] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.164124 20178 raft_consensus.cc:493] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.164517 20178 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.162302 19203 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:38.166206 20221 raft_consensus.cc:2804] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.166695 20221 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.166891 20206 raft_consensus.cc:2749] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.167721 20159 raft_consensus.cc:2749] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.167119 20221 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.168363 19203 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.173667 19157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
W20260709 12:14:38.177804 19203 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.178932 19203 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:38.179942 20159 raft_consensus.cc:2749] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:38.181063 19055 leader_election.cc:336] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.182857 20227 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.183499 20227 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.186532 20227 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.190594 20213 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.191334 20213 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:38.192849 19129 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.193783 20213 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.194950 20222 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.180002 19204 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:38.195564 20222 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.196455 20206 raft_consensus.cc:2749] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.198050 20222 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.201906 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.202801 19236 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:38.203336 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.204066 19239 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:38.204188 19056 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.205686 20208 raft_consensus.cc:2804] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.206094 20208 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.206471 20208 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.207692 19055 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.105247 20204 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.211000 19056 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.216445 20207 raft_consensus.cc:2804] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.216946 20207 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.217260 20211 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.217947 20211 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:38.218871 19055 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.220551 20211 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373)
I20260709 12:14:38.221698 20209 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.224217 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.225020 19239 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:38.226217 20222 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.227017 20222 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.228924 20221 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.217329 20207 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.232038 20222 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:38.232406 20227 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.235733 20227 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.236374 18911 catalog_manager.cc:5697] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.237861 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.241216 20227 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.241930 19130 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.243700 20227 raft_consensus.cc:2804] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.245749 20227 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.246307 20227 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.261998 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bee0ab48d58247cf8e32af4e5a43e66a" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:38.262523 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.262795 19239 raft_consensus.cc:3060] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.253593 20208 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.260810 19157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:38.258766 20226 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.245121 20222 raft_consensus.cc:493] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.246728 20219 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.248884 20209 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:38.262921 19129 leader_election.cc:336] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.266306 20226 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.266815 20222 raft_consensus.cc:515] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.282976 20221 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.267177 20219 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.279748 20226 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.285786 20219 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.287241 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.287966 20222 leader_election.cc:290] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:38.288156 19088 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 3.
I20260709 12:14:38.289767 19204 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.271395 20178 raft_consensus.cc:515] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.290179 19130 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.281004 20205 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.291633 20205 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:38.292480 20226 raft_consensus.cc:2749] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.293990 20205 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.266206 20208 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.290593 19239 raft_consensus.cc:2468] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 3.
I20260709 12:14:38.296309 20178 leader_election.cc:290] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:38.299077 19055 leader_election.cc:336] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.302354 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ec43e6601ae4d8cab4e09c7e77632ae" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:38.303164 19086 raft_consensus.cc:3060] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.303987 19056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.68:33373, user_credentials={real_user=slave}} blocked reactor thread for 67351.1us
I20260709 12:14:38.309648 20178 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.310372 20178 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.301668 20219 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:38.314916 20219 raft_consensus.cc:493] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.315409 20219 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:38.314275 19055 leader_election.cc:336] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.316536 19055 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:38.317651 20211 raft_consensus.cc:2749] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:38.319350 19055 leader_election.cc:336] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.326020 19088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
W20260709 12:14:38.326465 19129 leader_election.cc:336] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.326874 19088 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 3.
W20260709 12:14:38.327155 19129 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.312935 20178 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.328316 19204 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.330397 20178 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:38.330868 20178 raft_consensus.cc:493] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.331311 20178 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:38.332916 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0fdf1bfb7c54f7da0aa45218c041b08" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:38.333807 19239 raft_consensus.cc:3060] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.325026 20207 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.336714 19236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d792b44d6c54e1490e101d60dbebb82" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
W20260709 12:14:38.337136 19055 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.337270 20207 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.337491 19236 raft_consensus.cc:3060] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.339972 19156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
W20260709 12:14:38.340016 19055 leader_election.cc:336] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.340783 19156 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 2.
I20260709 12:14:38.347052 19086 raft_consensus.cc:2468] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 3.
I20260709 12:14:38.348390 19204 leader_election.cc:304] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.348489 20227 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.349522 20204 raft_consensus.cc:2804] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.350427 20204 raft_consensus.cc:697] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:38.352990 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" is_pre_election: true
I20260709 12:14:38.353824 19237 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:38.354480 19130 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.356089 20216 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.356101 19238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9061504a72094d2abbe26a00b4998e9c" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:38.357466 19238 raft_consensus.cc:3060] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.342795 19056 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.360483 20207 raft_consensus.cc:2749] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:38.356714 20216 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.363260 20204 consensus_queue.cc:237] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.365955 20216 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.356642 20221 raft_consensus.cc:2804] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.367822 20221 raft_consensus.cc:697] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:38.368912 20221 consensus_queue.cc:237] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.370278 19237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee5a381266c643c9a8b1ea339f87b328" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:38.371228 19237 raft_consensus.cc:3060] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.378450 19238 raft_consensus.cc:2468] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 3.
W20260709 12:14:38.379786 19055 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.384455 19055 leader_election.cc:304] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.385424 20211 raft_consensus.cc:2804] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.385848 20211 raft_consensus.cc:493] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.386591 20211 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.386816 20227 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.385838 19156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:38.391417 19156 raft_consensus.cc:2468] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
I20260709 12:14:38.399353 20205 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.400223 20205 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.402112 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561" is_pre_election: true
I20260709 12:14:38.402959 19086 raft_consensus.cc:2468] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 2.
I20260709 12:14:38.403025 20205 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.406769 19130 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.407876 20216 raft_consensus.cc:2804] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.408380 20216 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.408838 20216 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.409617 19130 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:38.411540 20226 raft_consensus.cc:2804] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.412124 20226 raft_consensus.cc:697] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:38.411670 20178 raft_consensus.cc:515] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:38.413170 20226 consensus_queue.cc:237] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.400151 20219 raft_consensus.cc:515] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.417171 19239 raft_consensus.cc:2468] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 3.
I20260709 12:14:38.446215 19056 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.428547 19203 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:38.415975 19086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbccd6116cc5452ebe3b6f446963bcd2" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:38.420919 19130 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:38.448752 19086 raft_consensus.cc:3060] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:38.448961 19129 leader_election.cc:336] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.449208 20207 raft_consensus.cc:2804] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.427461 20219 leader_election.cc:290] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.427901 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d99ec37464bd4047a9485e16e72301e9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:38.400147 20211 raft_consensus.cc:515] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.450752 19087 raft_consensus.cc:3060] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:38.452601 20211 leader_election.cc:290] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:38.452836 20204 raft_consensus.cc:2804] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.453249 20207 raft_consensus.cc:697] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:38.453317 20204 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.453931 20204 raft_consensus.cc:3060] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.454579 19129 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.454420 20207 consensus_queue.cc:237] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.442484 18909 catalog_manager.cc:5697] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:38.433928 19236 raft_consensus.cc:2468] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 3.
I20260709 12:14:38.462890 19056 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
W20260709 12:14:38.463928 19203 leader_election.cc:336] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:38.464926 19203 leader_election.cc:336] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.434906 19237 raft_consensus.cc:2468] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 3.
W20260709 12:14:38.466456 19203 leader_election.cc:336] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.468128 20227 raft_consensus.cc:2804] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.468596 20227 raft_consensus.cc:493] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.468735 20211 raft_consensus.cc:2804] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.469007 20227 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.469411 20211 raft_consensus.cc:697] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:38.470876 20211 consensus_queue.cc:237] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.467458 19086 raft_consensus.cc:2468] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 4.
I20260709 12:14:38.475956 19204 leader_election.cc:304] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.475980 19056 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.451268 18910 catalog_manager.cc:5697] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 3 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:38.482455 19055 leader_election.cc:336] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.483347 20213 raft_consensus.cc:2804] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.414517 20178 leader_election.cc:290] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.483970 20213 raft_consensus.cc:697] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:38.485174 20213 consensus_queue.cc:237] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.482713 20219 raft_consensus.cc:2804] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:38.492770 20219 raft_consensus.cc:697] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:38.493894 20219 consensus_queue.cc:237] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:38.497067 20204 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.499442 19156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df92e505d2454dea8eabdad38c8f36e4" candidate_uuid: "666fef4f7e5a4c32813dad8dfade8561" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:38.499716 20204 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.503490 19156 raft_consensus.cc:3060] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.493557 19087 raft_consensus.cc:2468] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 4.
I20260709 12:14:38.506963 20227 raft_consensus.cc:515] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.511057 19204 leader_election.cc:304] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 666fef4f7e5a4c32813dad8dfade8561; no voters: 
I20260709 12:14:38.508769 18911 catalog_manager.cc:5697] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:38.513671 20178 raft_consensus.cc:2804] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:38.515417 20178 raft_consensus.cc:697] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:38.515973 20227 leader_election.cc:290] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Requested vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:38.516381 20178 consensus_queue.cc:237] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:38.519901 19156 raft_consensus.cc:2468] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 666fef4f7e5a4c32813dad8dfade8561 in term 3.
I20260709 12:14:38.520414 19087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edf86dd5411044549e507998d7b14952" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "666fef4f7e5a4c32813dad8dfade8561"
I20260709 12:14:38.521004 20216 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.522644 19087 raft_consensus.cc:3060] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:38.524397 20216 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Requested vote from peers 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68:33373), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.529958 19055 leader_election.cc:304] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.531016 20211 raft_consensus.cc:2804] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:14:38.531141 19203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:43327, user_credentials={real_user=slave}} blocked reactor thread for 64695.5us
I20260709 12:14:38.531639 20211 raft_consensus.cc:697] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Becoming Leader. State: Replica: 666fef4f7e5a4c32813dad8dfade8561, State: Running, Role: LEADER
I20260709 12:14:38.532668 20211 consensus_queue.cc:237] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.535352 19237 raft_consensus.cc:1275] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Refusing update from remote peer 666fef4f7e5a4c32813dad8dfade8561: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:38.540416 20213 consensus_queue.cc:1048] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:38.542310 19087 raft_consensus.cc:2468] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cab1cfd695145f992765d39dbe662c7 in term 3.
I20260709 12:14:38.544848  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
I20260709 12:14:38.548704 19159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
W20260709 12:14:38.543089 19129 leader_election.cc:336] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.554518 19159 raft_consensus.cc:2393] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c0b3983ae6c48108f6f3632116cf5a8 in current term 3: Already voted for candidate 6cab1cfd695145f992765d39dbe662c7 in this term.
I20260709 12:14:38.557564 19130 leader_election.cc:304] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.559803 20216 raft_consensus.cc:2804] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.560492 20216 raft_consensus.cc:697] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: Becoming Leader. State: Replica: 6cab1cfd695145f992765d39dbe662c7, State: Running, Role: LEADER
I20260709 12:14:38.554780 19156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7" is_pre_election: true
I20260709 12:14:38.561664 20216 consensus_queue.cc:237] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } }
I20260709 12:14:38.562602 19156 raft_consensus.cc:2468] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 2.
W20260709 12:14:38.564136 19129 leader_election.cc:336] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:38.584164 19130 consensus_peers.cc:597] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.607503 19239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "169518fbf85c4378af0ff5af5df6dff1" candidate_uuid: "6cab1cfd695145f992765d39dbe662c7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2c0b3983ae6c48108f6f3632116cf5a8"
I20260709 12:14:38.608846 19239 raft_consensus.cc:2393] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6cab1cfd695145f992765d39dbe662c7 in current term 3: Already voted for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in this term.
W20260709 12:14:38.610083 19203 leader_election.cc:336] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.578615 18910 catalog_manager.cc:5697] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.615540 18910 catalog_manager.cc:5697] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.640774 18911 catalog_manager.cc:5697] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "6cab1cfd695145f992765d39dbe662c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.617923 18910 catalog_manager.cc:5697] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:38.663525 19055 consensus_peers.cc:597] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.669241 19130 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cab1cfd695145f992765d39dbe662c7; no voters: 2c0b3983ae6c48108f6f3632116cf5a8, 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.662956 18909 catalog_manager.cc:5697] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 3 to 4, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 4 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.673111 19203 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 
I20260709 12:14:38.672423 20216 raft_consensus.cc:2749] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.674970 20205 raft_consensus.cc:2804] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:38.673350 18909 catalog_manager.cc:5697] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 3 to 4, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 4 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:38.677567 20205 raft_consensus.cc:493] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3fcac1aec8df4c599b4c67550ce347e1)
I20260709 12:14:38.690336 20205 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.683045 19204 consensus_peers.cc:597] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.685593 19203 leader_election.cc:336] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:38.695176 19204 consensus_peers.cc:597] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.700659 19203 leader_election.cc:336] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:38.701952 19203 leader_election.cc:336] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:38.703397 19203 leader_election.cc:336] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.703816 19203 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:38.704584 20178 raft_consensus.cc:2749] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:38.703850 18910 catalog_manager.cc:5697] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66). New cstate: current_term: 3 leader_uuid: "666fef4f7e5a4c32813dad8dfade8561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:38.720073 19130 consensus_peers.cc:597] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.722424 20205 raft_consensus.cc:515] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.725039 20205 leader_election.cc:290] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Requested vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:38.726755 19159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e913e5c22bd646e1944cf0bac9dae4c9" candidate_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6cab1cfd695145f992765d39dbe662c7"
I20260709 12:14:38.727623 19159 raft_consensus.cc:3060] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:38.731490 19203 leader_election.cc:336] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:38.733704  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:14:38.735677 19204 consensus_peers.cc:597] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.740108  9261 tablet_replica.cc:333] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.741358  9261 raft_consensus.cc:2243] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.742444  9261 pending_rounds.cc:70] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: Trying to abort 1 pending ops.
I20260709 12:14:38.742683  9261 pending_rounds.cc:77] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622644688556032 op_type: NO_OP noop_request { }
I20260709 12:14:38.743057  9261 raft_consensus.cc:2889] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:38.743396  9261 raft_consensus.cc:2272] T ee5a381266c643c9a8b1ea339f87b328 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.745918 19159 raft_consensus.cc:2468] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c0b3983ae6c48108f6f3632116cf5a8 in term 3.
I20260709 12:14:38.746753  9261 tablet_replica.cc:333] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.748217  9261 raft_consensus.cc:2243] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.748991 19203 leader_election.cc:304] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8, 6cab1cfd695145f992765d39dbe662c7; no voters: 3fcac1aec8df4c599b4c67550ce347e1
I20260709 12:14:38.749415  9261 raft_consensus.cc:2272] T 04add48cb044473fbc449788540f24c9 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.750633 20205 raft_consensus.cc:2804] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.751259 20205 raft_consensus.cc:697] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 LEADER]: Becoming Leader. State: Replica: 2c0b3983ae6c48108f6f3632116cf5a8, State: Running, Role: LEADER
I20260709 12:14:38.753329  9261 tablet_replica.cc:333] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.752863 20205 consensus_queue.cc:237] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:38.756999  9261 raft_consensus.cc:2243] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.758009  9261 pending_rounds.cc:70] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: Trying to abort 1 pending ops.
I20260709 12:14:38.758288  9261 pending_rounds.cc:77] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622644574781440 op_type: NO_OP noop_request { }
I20260709 12:14:38.758682  9261 raft_consensus.cc:2889] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:38.759105  9261 raft_consensus.cc:2272] T c0fdf1bfb7c54f7da0aa45218c041b08 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.763485  9261 tablet_replica.cc:333] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.764586  9261 raft_consensus.cc:2243] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.765196  9261 raft_consensus.cc:2272] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.774055  9261 tablet_replica.cc:333] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.775247  9261 raft_consensus.cc:2243] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:14:38.775871 19130 consensus_peers.cc:597] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.776319  9261 raft_consensus.cc:2272] T 718f20f68825449c836f67bd5a4bae55 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.777037 18909 catalog_manager.cc:5697] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 reported cstate change: term changed from 2 to 3, leader changed from 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65) to 2c0b3983ae6c48108f6f3632116cf5a8 (127.9.11.68). New cstate: current_term: 3 leader_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:38.780036  9261 tablet_replica.cc:333] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
W20260709 12:14:38.781826 19204 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.783198  9261 raft_consensus.cc:2243] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:38.784286 19129 consensus_peers.cc:597] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.785080  9261 raft_consensus.cc:2272] T 62d133466fbe4df8a6e09156cec6acf4 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.788869  9261 tablet_replica.cc:333] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.790274  9261 raft_consensus.cc:2243] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.791986  9261 raft_consensus.cc:2272] T e8e8cd17ef794dd18367b9052aae4c29 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.795279  9261 tablet_replica.cc:333] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.796240  9261 raft_consensus.cc:2243] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.796857  9261 raft_consensus.cc:2272] T 0ccbbf4c1622458faa667dc368b16eff P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.797758 19204 consensus_peers.cc:597] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.799930  9261 tablet_replica.cc:333] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.800668  9261 raft_consensus.cc:2243] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.801792  9261 raft_consensus.cc:2272] T 0eed4a045f6747428301e56cf20644b7 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.804992  9261 tablet_replica.cc:333] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.805662  9261 raft_consensus.cc:2243] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.806141  9261 raft_consensus.cc:2272] T 20a717f41e3045c8b600ca072c4b1efd P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.809262  9261 tablet_replica.cc:333] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.810145  9261 raft_consensus.cc:2243] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.810770  9261 raft_consensus.cc:2272] T eff579fc38a94049a77efcb07b6bf35c P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.811462 19204 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.814050  9261 tablet_replica.cc:333] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.815119  9261 raft_consensus.cc:2243] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.816037  9261 raft_consensus.cc:2272] T 5373fb94420c4369bab88209ddbd5324 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.818214  9261 tablet_replica.cc:333] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.818856  9261 raft_consensus.cc:2243] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.819585  9261 pending_rounds.cc:70] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: Trying to abort 1 pending ops.
I20260709 12:14:38.819801  9261 pending_rounds.cc:77] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622644895784960 op_type: NO_OP noop_request { }
I20260709 12:14:38.820101  9261 raft_consensus.cc:2889] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:38.820381  9261 raft_consensus.cc:2272] T df92e505d2454dea8eabdad38c8f36e4 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.822759  9261 tablet_replica.cc:333] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.823488  9261 raft_consensus.cc:2243] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.823995  9261 raft_consensus.cc:2272] T 7a1299cc26f84cbaa6278121975f7961 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.826184  9261 tablet_replica.cc:333] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.826830  9261 raft_consensus.cc:2243] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.827777  9261 raft_consensus.cc:2272] T 9c7af1ca05aa414999a70c06b637cb56 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.830039  9261 tablet_replica.cc:333] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.830677  9261 raft_consensus.cc:2243] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.831244  9261 raft_consensus.cc:2272] T b9af570d255149c5aedd8856d3698e12 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.834326  9261 tablet_replica.cc:333] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.835067  9261 raft_consensus.cc:2243] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.835745  9261 pending_rounds.cc:70] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: Trying to abort 1 pending ops.
I20260709 12:14:38.835966  9261 pending_rounds.cc:77] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622644627456000 op_type: NO_OP noop_request { }
I20260709 12:14:38.836269  9261 raft_consensus.cc:2889] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:38.836557  9261 raft_consensus.cc:2272] T 5d792b44d6c54e1490e101d60dbebb82 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.839746  9261 tablet_replica.cc:333] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.840409  9261 raft_consensus.cc:2243] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.841378  9261 raft_consensus.cc:2272] T 428f9c2176004b6aa007c6a7af377fcb P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.843778  9261 tablet_replica.cc:333] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.844391  9261 raft_consensus.cc:2243] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.845373  9261 raft_consensus.cc:2272] T caf9e5cd30214f99a71d1f25f34dae84 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.847592  9261 tablet_replica.cc:333] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.848414  9261 raft_consensus.cc:2243] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.849049  9261 raft_consensus.cc:2272] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.851811  9261 tablet_replica.cc:333] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.852519  9261 raft_consensus.cc:2243] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.853406  9261 raft_consensus.cc:2272] T 24dc0ead3da3436d97931d4aff80a357 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.853937 19186 debug-util.cc:398] Leaking SignalData structure 0x7b08000852c0 after lost signal to thread 9264
W20260709 12:14:38.855309 19186 debug-util.cc:398] Leaking SignalData structure 0x7b080002f3c0 after lost signal to thread 18947
I20260709 12:14:38.856027  9261 tablet_replica.cc:333] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
W20260709 12:14:38.856793 19186 debug-util.cc:398] Leaking SignalData structure 0x7b0800029b00 after lost signal to thread 19115
I20260709 12:14:38.856918  9261 raft_consensus.cc:2243] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:14:38.858142 19186 debug-util.cc:398] Leaking SignalData structure 0x7b080020cea0 after lost signal to thread 19189
I20260709 12:14:38.858207  9261 raft_consensus.cc:2272] T 55dd90070791474ea046bd86b13e3f75 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.859477 19186 debug-util.cc:398] Leaking SignalData structure 0x7b080024ef60 after lost signal to thread 19263
I20260709 12:14:38.861472  9261 tablet_replica.cc:333] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.862398  9261 raft_consensus.cc:2243] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.863672  9261 raft_consensus.cc:2272] T 67045c0c43a64d4a896e1928eda8289f P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.866560  9261 tablet_replica.cc:333] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.867411  9261 raft_consensus.cc:2243] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.868117  9261 raft_consensus.cc:2272] T 43888d54ff1d470a8d63fb846d1d9282 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.868616 19204 consensus_peers.cc:597] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.871722  9261 tablet_replica.cc:333] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.872697  9261 raft_consensus.cc:2243] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.873840  9261 raft_consensus.cc:2272] T c3afa3fafa244643a6da11e31e841d34 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.877000  9261 tablet_replica.cc:333] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.877909  9261 raft_consensus.cc:2243] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:38.878224 19129 consensus_peers.cc:597] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.878474  9261 raft_consensus.cc:2272] T 489f2ef3c1464126b44ded0df97dbee1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.879503 19204 consensus_peers.cc:597] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.881471  9261 tablet_replica.cc:333] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.882463  9261 raft_consensus.cc:2243] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.883090  9261 raft_consensus.cc:2272] T 54df033d2cbe422390e0fa4bf287ee14 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.886243  9261 tablet_replica.cc:333] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.887164  9261 raft_consensus.cc:2243] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.887867  9261 raft_consensus.cc:2272] T d99ec37464bd4047a9485e16e72301e9 P 666fef4f7e5a4c32813dad8dfade8561 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.891040  9261 tablet_replica.cc:333] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.891958  9261 raft_consensus.cc:2243] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.893186  9261 raft_consensus.cc:2272] T 0741f1cc27bd423d9b9c55f4665cfce1 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.896502  9261 tablet_replica.cc:333] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.897394  9261 raft_consensus.cc:2243] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.897442 19236 raft_consensus.cc:1275] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:38.898205  9261 raft_consensus.cc:2272] T d409ffd2084d48bf8886755a2146b5e0 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.898970 20216 consensus_queue.cc:1048] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:38.901744  9261 tablet_replica.cc:333] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.902820  9261 raft_consensus.cc:2243] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.903533  9261 raft_consensus.cc:2272] T 9c4c556a691543529aee3c07a4015d8d P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.905550 19237 raft_consensus.cc:1275] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Refusing update from remote peer 6cab1cfd695145f992765d39dbe662c7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:38.906738  9261 tablet_replica.cc:333] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.907742  9261 raft_consensus.cc:2243] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.909205  9261 raft_consensus.cc:2272] T c890b118e3b14e4291cb99b7c9f31ae2 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.908957 20226 consensus_queue.cc:1048] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:38.913321  9261 tablet_replica.cc:333] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
W20260709 12:14:38.910923 19130 consensus_peers.cc:597] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.915719  9261 raft_consensus.cc:2243] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.916407  9261 raft_consensus.cc:2272] T c394ca853d394fdbafaa3a25b2633648 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.919850  9261 tablet_replica.cc:333] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
W20260709 12:14:38.920413 19130 consensus_peers.cc:597] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.920809  9261 raft_consensus.cc:2243] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:14:38.927325 19204 consensus_peers.cc:597] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.942646  9261 raft_consensus.cc:2272] T f9eff763afc14d8aa7bbd4505c26d0c3 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.953706 19204 consensus_peers.cc:597] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.955914 19130 consensus_peers.cc:597] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.959312 19130 consensus_peers.cc:597] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.959909 19130 consensus_peers.cc:597] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.960848  9261 tablet_replica.cc:333] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.962532  9261 raft_consensus.cc:2243] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.969522  9261 raft_consensus.cc:2272] T fbccd6116cc5452ebe3b6f446963bcd2 P 666fef4f7e5a4c32813dad8dfade8561 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.962772 19130 consensus_peers.cc:597] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.974768  9261 tablet_replica.cc:333] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.975883  9261 raft_consensus.cc:2243] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.976542  9261 raft_consensus.cc:2272] T 20ddebbef4404377b6b25cc37866b9e2 P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.982147  9261 tablet_replica.cc:333] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.983109  9261 raft_consensus.cc:2243] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.984324  9261 raft_consensus.cc:2272] T d311e304cfc84db5a4dfc30459f4d72e P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:38.990350  9261 tablet_replica.cc:333] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.991298  9261 raft_consensus.cc:2243] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:38.992488  9261 raft_consensus.cc:2272] T 0cc2ff8089f3493a8d2e92e4deb28750 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:38.993399 19130 consensus_peers.cc:597] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.995819  9261 tablet_replica.cc:333] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:38.997219  9261 raft_consensus.cc:2243] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:38.997802  9261 raft_consensus.cc:2272] T edf86dd5411044549e507998d7b14952 P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.000979  9261 tablet_replica.cc:333] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:39.001821  9261 raft_consensus.cc:2243] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.002394  9261 raft_consensus.cc:2272] T 3d0ebf6b04c74f4c994950e78fccf2fe P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.005623  9261 tablet_replica.cc:333] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:39.006466  9261 raft_consensus.cc:2243] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:14:39.012168 19203 consensus_peers.cc:597] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.014128  9261 raft_consensus.cc:2272] T 8747727903d14849a4d18d8676bbf505 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.017921  9261 tablet_replica.cc:333] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:39.018697  9261 raft_consensus.cc:2243] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.019284  9261 raft_consensus.cc:2272] T 2b5f4455e6964b99b522e009ca88b6d8 P 666fef4f7e5a4c32813dad8dfade8561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.023433  9261 tablet_replica.cc:333] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:39.024103  9261 raft_consensus.cc:2243] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.024677  9261 raft_consensus.cc:2272] T 9ec43e6601ae4d8cab4e09c7e77632ae P 666fef4f7e5a4c32813dad8dfade8561 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.027235  9261 tablet_replica.cc:333] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561: stopping tablet replica
I20260709 12:14:39.027947  9261 raft_consensus.cc:2243] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.028548  9261 raft_consensus.cc:2272] T 64a4de1a07654315ae7678c1f9e8fcca P 666fef4f7e5a4c32813dad8dfade8561 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.029385 19130 consensus_peers.cc:597] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.060842 19204 consensus_peers.cc:597] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.071799 19129 consensus_peers.cc:597] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.113821 19203 consensus_peers.cc:597] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.142171  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
W20260709 12:14:39.149766 19203 consensus_peers.cc:597] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.156617 19130 consensus_peers.cc:597] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.162137 19130 consensus_peers.cc:597] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.167058 19203 consensus_peers.cc:597] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.245657 19156 raft_consensus.cc:1275] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Refusing update from remote peer 2c0b3983ae6c48108f6f3632116cf5a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:39.249308 20205 consensus_queue.cc:1048] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:14:39.319577  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
W20260709 12:14:39.327792 19203 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:39.342360 19203 consensus_peers.cc:597] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.390566 19203 consensus_peers.cc:597] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.432548 19203 consensus_peers.cc:597] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.458408  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:39.459798  9261 tablet_replica.cc:333] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.460656  9261 raft_consensus.cc:2243] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.461207  9261 raft_consensus.cc:2272] T 43f821d9236644bcb6676552b6ee4019 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.464244  9261 tablet_replica.cc:333] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.465313  9261 raft_consensus.cc:2243] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.466575  9261 raft_consensus.cc:2272] T 04add48cb044473fbc449788540f24c9 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.470062  9261 tablet_replica.cc:333] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.471066  9261 raft_consensus.cc:2243] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.471676  9261 raft_consensus.cc:2272] T 3cfc76678bd1474fb6f9826360c9e870 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.474763 19203 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:39.475037  9261 tablet_replica.cc:333] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
W20260709 12:14:39.483256 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:39.484881  9261 raft_consensus.cc:2243] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.485610  9261 raft_consensus.cc:2272] T 67045c0c43a64d4a896e1928eda8289f P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.487485 19203 consensus_peers.cc:597] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.490662  9261 tablet_replica.cc:333] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.491823  9261 raft_consensus.cc:2243] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.492449  9261 raft_consensus.cc:2272] T 6079f38db31043d2bc846667f08d7d1b P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.495684  9261 tablet_replica.cc:333] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.496733  9261 raft_consensus.cc:2243] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.498109  9261 raft_consensus.cc:2272] T eff579fc38a94049a77efcb07b6bf35c P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.501401  9261 tablet_replica.cc:333] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.502352  9261 raft_consensus.cc:2243] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.502967  9261 raft_consensus.cc:2272] T 8747727903d14849a4d18d8676bbf505 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.504249 19203 consensus_peers.cc:597] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:39.506099  9261 tablet_replica.cc:333] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.506863  9261 raft_consensus.cc:2243] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.508999  9261 raft_consensus.cc:2272] T 2b5f4455e6964b99b522e009ca88b6d8 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.511452  9261 tablet_replica.cc:333] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.512213  9261 raft_consensus.cc:2243] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.513111  9261 raft_consensus.cc:2272] T 9061504a72094d2abbe26a00b4998e9c P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.516258  9261 tablet_replica.cc:333] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.516906  9261 raft_consensus.cc:2243] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.517910  9261 raft_consensus.cc:2272] T bee0ab48d58247cf8e32af4e5a43e66a P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.521219  9261 tablet_replica.cc:333] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
W20260709 12:14:39.521948 19203 consensus_peers.cc:597] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.522060  9261 raft_consensus.cc:2243] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.522960  9261 raft_consensus.cc:2272] T 0ccbbf4c1622458faa667dc368b16eff P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.526122  9261 tablet_replica.cc:333] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.527082  9261 raft_consensus.cc:2243] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.527690  9261 raft_consensus.cc:2272] T 0eed4a045f6747428301e56cf20644b7 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.531231  9261 tablet_replica.cc:333] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.532260  9261 raft_consensus.cc:2243] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.533591  9261 raft_consensus.cc:2272] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.537235  9261 tablet_replica.cc:333] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.538192  9261 raft_consensus.cc:2243] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.539489  9261 raft_consensus.cc:2272] T e8e8cd17ef794dd18367b9052aae4c29 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.542815  9261 tablet_replica.cc:333] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.543774  9261 raft_consensus.cc:2243] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.544411  9261 raft_consensus.cc:2272] T 5ecdce0779954a328062dfca1a0364e4 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.547753  9261 tablet_replica.cc:333] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.548661  9261 raft_consensus.cc:2243] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.549285  9261 raft_consensus.cc:2272] T 24dc0ead3da3436d97931d4aff80a357 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.552477  9261 tablet_replica.cc:333] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.553355  9261 raft_consensus.cc:2243] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.553979  9261 raft_consensus.cc:2272] T 7726c731410a492188ef1b403a54f3bc P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.557283  9261 tablet_replica.cc:333] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.558219  9261 raft_consensus.cc:2243] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.558845  9261 raft_consensus.cc:2272] T 55dd90070791474ea046bd86b13e3f75 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.562110  9261 tablet_replica.cc:333] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.563095  9261 raft_consensus.cc:2243] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.563782  9261 raft_consensus.cc:2272] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.567153  9261 tablet_replica.cc:333] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.568077  9261 raft_consensus.cc:2243] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.568745  9261 raft_consensus.cc:2272] T c3afa3fafa244643a6da11e31e841d34 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.570715 19203 consensus_peers.cc:597] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.571962  9261 tablet_replica.cc:333] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.573086  9261 raft_consensus.cc:2243] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.573796  9261 raft_consensus.cc:2272] T caf9e5cd30214f99a71d1f25f34dae84 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.577415  9261 tablet_replica.cc:333] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.578390  9261 raft_consensus.cc:2243] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.579056  9261 raft_consensus.cc:2272] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.579254 19203 consensus_peers.cc:597] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.582286  9261 tablet_replica.cc:333] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.583402  9261 raft_consensus.cc:2243] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.584090  9261 raft_consensus.cc:2272] T 3d1280ae7e994dc59cf16f8ddef18b72 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.587301  9261 tablet_replica.cc:333] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.588274  9261 raft_consensus.cc:2243] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.588984  9261 raft_consensus.cc:2272] T 489f2ef3c1464126b44ded0df97dbee1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.592738  9261 tablet_replica.cc:333] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.593739  9261 raft_consensus.cc:2243] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.595067  9261 raft_consensus.cc:2272] T 54df033d2cbe422390e0fa4bf287ee14 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.598450  9261 tablet_replica.cc:333] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.599413  9261 raft_consensus.cc:2243] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.600085  9261 raft_consensus.cc:2272] T 0741f1cc27bd423d9b9c55f4665cfce1 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.603578  9261 tablet_replica.cc:333] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.604481  9261 raft_consensus.cc:2243] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.605067  9261 pending_rounds.cc:70] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: Trying to abort 1 pending ops.
I20260709 12:14:39.605305  9261 pending_rounds.cc:77] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622645803044864 op_type: NO_OP noop_request { }
I20260709 12:14:39.605693  9261 raft_consensus.cc:2889] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:39.606056  9261 raft_consensus.cc:2272] T e913e5c22bd646e1944cf0bac9dae4c9 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.609835  9261 tablet_replica.cc:333] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.610831  9261 raft_consensus.cc:2243] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:39.611393 19203 consensus_peers.cc:597] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:39.611585  9261 raft_consensus.cc:2272] T 0cc2ff8089f3493a8d2e92e4deb28750 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.612385 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.615351  9261 tablet_replica.cc:333] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.616464  9261 raft_consensus.cc:2243] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.617163  9261 raft_consensus.cc:2272] T df92e505d2454dea8eabdad38c8f36e4 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.620604  9261 tablet_replica.cc:333] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.621577  9261 raft_consensus.cc:2243] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:14:39.621944 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:39.622792  9261 raft_consensus.cc:2272] T 7a1299cc26f84cbaa6278121975f7961 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.626119  9261 tablet_replica.cc:333] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.627027  9261 raft_consensus.cc:2243] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.628273  9261 raft_consensus.cc:2272] T 64a4de1a07654315ae7678c1f9e8fcca P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.631518  9261 tablet_replica.cc:333] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.632470  9261 raft_consensus.cc:2243] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.633126  9261 raft_consensus.cc:2272] T a377e1ad0954438589b509e2491c8a72 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.636526  9261 tablet_replica.cc:333] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.637529  9261 raft_consensus.cc:2243] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.638896  9261 pending_rounds.cc:70] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: Trying to abort 1 pending ops.
I20260709 12:14:39.639225  9261 pending_rounds.cc:77] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622645012549632 op_type: NO_OP noop_request { }
I20260709 12:14:39.639631  9261 raft_consensus.cc:2889] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:39.640013  9261 raft_consensus.cc:2272] T edf86dd5411044549e507998d7b14952 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.643426  9261 tablet_replica.cc:333] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.644325  9261 raft_consensus.cc:2243] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.645027  9261 raft_consensus.cc:2272] T 169518fbf85c4378af0ff5af5df6dff1 P 6cab1cfd695145f992765d39dbe662c7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.648727  9261 tablet_replica.cc:333] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.649669  9261 raft_consensus.cc:2243] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.650965  9261 raft_consensus.cc:2272] T 9c4c556a691543529aee3c07a4015d8d P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.654093  9261 tablet_replica.cc:333] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.654950  9261 raft_consensus.cc:2243] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.656155  9261 raft_consensus.cc:2272] T d409ffd2084d48bf8886755a2146b5e0 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.659384  9261 tablet_replica.cc:333] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.660282  9261 raft_consensus.cc:2243] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.660944  9261 raft_consensus.cc:2272] T f9eff763afc14d8aa7bbd4505c26d0c3 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.664304  9261 tablet_replica.cc:333] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.665477  9261 raft_consensus.cc:2243] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.666939  9261 raft_consensus.cc:2272] T 20ddebbef4404377b6b25cc37866b9e2 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.670343  9261 tablet_replica.cc:333] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.671223  9261 raft_consensus.cc:2243] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.672492  9261 raft_consensus.cc:2272] T 2ff6615d74da465f9105797edb9c3698 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.675949  9261 tablet_replica.cc:333] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.676823  9261 raft_consensus.cc:2243] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.678172  9261 raft_consensus.cc:2272] T 3d0ebf6b04c74f4c994950e78fccf2fe P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.681826  9261 tablet_replica.cc:333] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.682771  9261 raft_consensus.cc:2243] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.684182  9261 raft_consensus.cc:2272] T 81a0d1ae8dd1431d9f241ba27f35f18f P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.687690  9261 tablet_replica.cc:333] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.688512  9261 raft_consensus.cc:2243] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.689237  9261 raft_consensus.cc:2272] T b9af570d255149c5aedd8856d3698e12 P 6cab1cfd695145f992765d39dbe662c7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.692548  9261 tablet_replica.cc:333] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.693356  9261 raft_consensus.cc:2243] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:39.693986  9261 raft_consensus.cc:2272] T 9c7af1ca05aa414999a70c06b637cb56 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:39.697425  9261 tablet_replica.cc:333] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7: stopping tablet replica
I20260709 12:14:39.698233  9261 raft_consensus.cc:2243] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:39.699610  9261 raft_consensus.cc:2272] T c394ca853d394fdbafaa3a25b2633648 P 6cab1cfd695145f992765d39dbe662c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:39.700606 19203 consensus_peers.cc:597] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.718906 19203 consensus_peers.cc:597] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.786388 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:39.856376 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.885366 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:40.088828 20257 raft_consensus.cc:493] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.089462 20257 raft_consensus.cc:515] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.091591 20257 leader_election.cc:290] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.096099 19204 leader_election.cc:336] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.097692 19203 leader_election.cc:336] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.098091 19203 leader_election.cc:304] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.098934 20254 raft_consensus.cc:2749] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:40.102185 19203 consensus_peers.cc:597] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:40.104421 20204 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.105159 20204 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.112083 20204 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.121145 19204 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.127460 19203 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.128028 19203 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.128821 20204 raft_consensus.cc:2749] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:40.149178 19203 consensus_peers.cc:597] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:40.186430 20204 raft_consensus.cc:493] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.187060 20204 raft_consensus.cc:515] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.189234 20204 leader_election.cc:290] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.193740 19204 leader_election.cc:336] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.194319 19203 leader_election.cc:336] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.194782 19203 leader_election.cc:304] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.195597 20204 raft_consensus.cc:2749] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:40.211587 20257 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.212196 20257 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.214530 20257 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:40.215927 20257 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.216522 20257 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.218968 20257 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.221894 19204 leader_election.cc:336] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.222590 19203 leader_election.cc:336] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.223119 19203 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.223951 20257 raft_consensus.cc:2749] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:40.224953 19204 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.226271 19203 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.226660 19203 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.227334 20257 raft_consensus.cc:2749] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:40.275050 20257 raft_consensus.cc:493] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.275595 20257 raft_consensus.cc:515] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.277557 20257 leader_election.cc:290] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.282038 19204 leader_election.cc:336] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.282626 19203 leader_election.cc:336] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.283042 19203 leader_election.cc:304] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.283674 20257 raft_consensus.cc:2749] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:40.284106 20254 raft_consensus.cc:493] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.284672 20254 raft_consensus.cc:515] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.286489 20254 leader_election.cc:290] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:40.289486 20254 raft_consensus.cc:493] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.289839 20254 raft_consensus.cc:515] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:40.290580 19203 leader_election.cc:336] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.292006 20254 leader_election.cc:290] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.295246 19204 leader_election.cc:336] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.295768 19204 leader_election.cc:304] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.296684 20254 raft_consensus.cc:2749] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:40.297415 19203 leader_election.cc:336] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:40.299294 19204 leader_election.cc:336] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.299757 19204 leader_election.cc:304] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.301005 20254 raft_consensus.cc:2749] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:40.304114 20257 raft_consensus.cc:493] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.304610 20257 raft_consensus.cc:515] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.307552 20257 leader_election.cc:290] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.311542 19203 leader_election.cc:336] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:40.312034 19204 leader_election.cc:336] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.312497 19204 leader_election.cc:304] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.313230 20257 raft_consensus.cc:2749] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:40.314038 20257 raft_consensus.cc:493] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.314445 20257 raft_consensus.cc:515] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.316458 20257 leader_election.cc:290] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.327273 19204 leader_election.cc:336] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.328715 19203 leader_election.cc:336] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.329130 19203 leader_election.cc:304] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.331281 20254 raft_consensus.cc:2749] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:40.334400 20254 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.334932 20254 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.337343 20254 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.341251 19204 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.343436 19203 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.343879 19203 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.344676 20254 raft_consensus.cc:2749] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:40.376802 19203 consensus_peers.cc:597] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:40.383774 20254 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.384516 20254 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:40.386796 20254 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:40.387107 20257 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.387696 20257 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.393062 20257 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.393275 19203 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:40.394088 19203 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.394498 19203 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.395327 20257 raft_consensus.cc:2749] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:40.396382 19204 leader_election.cc:336] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.400262 19203 leader_election.cc:336] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.400730 19203 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.401507 20257 raft_consensus.cc:2749] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:40.425504 19203 consensus_peers.cc:597] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:40.443351 20257 raft_consensus.cc:493] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:40.443922 20257 raft_consensus.cc:515] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.446069 20257 leader_election.cc:290] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.450631 19204 leader_election.cc:336] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.453517 19203 leader_election.cc:336] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.454188 19203 leader_election.cc:304] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.456732 20254 raft_consensus.cc:2749] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:40.487951 20254 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:40.488646 20254 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:40.491359 20254 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:40.498974 19203 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:40.499905 19203 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.500334 19203 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.501240 20254 raft_consensus.cc:2749] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:40.713495 20257 raft_consensus.cc:493] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.714169 20257 raft_consensus.cc:515] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:40.716810 20257 leader_election.cc:290] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:40.720901 19203 leader_election.cc:336] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:40.721889 19203 leader_election.cc:336] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.722227 19203 leader_election.cc:304] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.722884 20257 raft_consensus.cc:2749] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:40.736717 20257 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.737334 20257 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.739737 20257 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.744923 19203 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:40.745558 19204 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.745994 19204 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.746634 20257 raft_consensus.cc:2749] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:40.760536 20257 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:40.761219 20257 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.763844 20257 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.768617 19204 leader_election.cc:336] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:40.769992 19203 leader_election.cc:336] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.770473 19203 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.771245 20257 raft_consensus.cc:2749] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:40.784231 20257 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.784757 20257 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:40.786654 20257 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:40.790596 19203 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:40.791214 19204 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.791685 19204 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:40.792439 20257 raft_consensus.cc:2749] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:40.803678 20257 raft_consensus.cc:493] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:40.804168 20257 raft_consensus.cc:515] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:40.806138 20257 leader_election.cc:290] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:40.808950 20257 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:40.809517 20257 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
W20260709 12:14:40.810024  9261 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 19186)
W20260709 12:14:40.811537 19203 leader_election.cc:336] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.811923 20257 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:40.813957 19203 leader_election.cc:336] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.814313 19203 leader_election.cc:304] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.814906 20257 raft_consensus.cc:2749] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:40.817080 19203 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:40.817682 19203 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.818068 19203 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.818728 20254 raft_consensus.cc:2749] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:40.843381 19204 consensus_peers.cc:597] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:40.882498 20254 raft_consensus.cc:493] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:40.883256 20254 raft_consensus.cc:515] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.885912 20254 leader_election.cc:290] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.889813 19203 leader_election.cc:336] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:40.890328 19204 leader_election.cc:336] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.890766 19204 leader_election.cc:304] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.891501 20254 raft_consensus.cc:2749] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:40.959215 20254 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:40.959785 20254 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:40.961772 20254 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:40.965197 19203 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:40.965950 20254 raft_consensus.cc:493] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
W20260709 12:14:40.965981 19203 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:40.966524 19203 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.966531 20254 raft_consensus.cc:515] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:40.967414 20257 raft_consensus.cc:2749] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:40.968816 20254 leader_election.cc:290] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:40.972204 19203 leader_election.cc:336] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:40.973121 19204 leader_election.cc:336] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:40.973620 19204 leader_election.cc:304] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:40.974434 20254 raft_consensus.cc:2749] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:41.017939 19204 consensus_peers.cc:597] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:41.144980 20257 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:41.145723 20257 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:41.148241 20257 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:41.151981 19203 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:41.152671 19203 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:41.153002 19203 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:41.153663 20257 raft_consensus.cc:2749] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:41.169643 19204 consensus_peers.cc:597] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.226959 19204 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.258563 19204 consensus_peers.cc:597] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.408059 19204 consensus_peers.cc:597] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.420739 19204 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.427673 19204 consensus_peers.cc:597] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.545886 19203 consensus_peers.cc:597] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.551376 19203 consensus_peers.cc:597] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.617666 19203 consensus_peers.cc:597] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.632568 19203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111) [suppressed 456 similar messages]
W20260709 12:14:41.639664 19203 consensus_peers.cc:597] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.640390 19204 consensus_peers.cc:597] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:41.707105 20254 raft_consensus.cc:493] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:41.707787 20254 raft_consensus.cc:515] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:41.710283 20254 leader_election.cc:290] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:41.714324 19203 leader_election.cc:336] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:41.714905 19204 leader_election.cc:336] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:41.715392 19204 leader_election.cc:304] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:41.716171 20254 raft_consensus.cc:2749] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:41.748881 19203 consensus_peers.cc:597] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:41.750303 19204 consensus_peers.cc:597] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.761019 19204 consensus_peers.cc:597] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:41.776683 20254 raft_consensus.cc:493] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:41.777355 20254 raft_consensus.cc:515] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:41.779868 20254 leader_election.cc:290] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:41.783041 19204 leader_election.cc:336] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:41.784157 19203 leader_election.cc:336] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:41.784544 19203 leader_election.cc:304] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:41.785202 20254 raft_consensus.cc:2749] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:41.810616  9261 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 19186)
W20260709 12:14:41.821012 19203 consensus_peers.cc:597] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.824716 19203 consensus_peers.cc:597] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:41.843166 20254 raft_consensus.cc:493] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:41.843910 20254 raft_consensus.cc:515] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:41.846524 20254 leader_election.cc:290] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:41.849850 19204 leader_election.cc:336] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:41.850715 19203 leader_election.cc:336] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:41.851189 19203 leader_election.cc:304] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:41.851989 20254 raft_consensus.cc:2749] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:41.901692 20254 raft_consensus.cc:493] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:41.902276 20254 raft_consensus.cc:515] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:41.904420 20254 leader_election.cc:290] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:41.908633 19204 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:41.909257 19203 leader_election.cc:336] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:41.909732 19203 leader_election.cc:304] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:41.910377 20254 raft_consensus.cc:2749] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:41.932400 19203 consensus_peers.cc:597] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.937270 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:41.968374 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:41.973282 19203 consensus_peers.cc:597] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:41.980543 20254 raft_consensus.cc:493] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:41.981204 20254 raft_consensus.cc:515] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:41.983700 20254 leader_election.cc:290] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:41.987986 19204 leader_election.cc:336] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:41.989667 19203 leader_election.cc:336] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:41.990123 19203 leader_election.cc:304] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:41.990911 20254 raft_consensus.cc:2749] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:42.020107 20254 raft_consensus.cc:493] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:42.020877 20254 raft_consensus.cc:515] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:42.023820 20254 leader_election.cc:290] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:42.026363 20254 raft_consensus.cc:493] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
W20260709 12:14:42.027195 19204 leader_election.cc:336] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.027024 20254 raft_consensus.cc:515] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:42.028385 19203 leader_election.cc:336] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:42.028767 19203 leader_election.cc:304] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.029399 20257 raft_consensus.cc:2749] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:42.030781 20254 leader_election.cc:290] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:42.033998 19203 leader_election.cc:336] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:42.034682 19204 leader_election.cc:336] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.035223 19204 leader_election.cc:304] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:42.035981 20257 raft_consensus.cc:2749] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:42.084110 19203 consensus_peers.cc:597] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:42.086352 19203 consensus_peers.cc:597] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:42.100557 20257 raft_consensus.cc:493] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:42.101367 20257 raft_consensus.cc:515] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:42.102378 19203 consensus_peers.cc:597] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:42.104264 20257 leader_election.cc:290] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:42.105317 19203 consensus_peers.cc:597] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.107950 19204 leader_election.cc:336] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:42.108657 19203 leader_election.cc:336] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:42.109097 19203 leader_election.cc:304] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.109894 20257 raft_consensus.cc:2749] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:42.115391 19203 consensus_peers.cc:597] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:42.152782 19203 consensus_peers.cc:597] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.160312 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.179214 19203 consensus_peers.cc:597] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.206820 19203 consensus_peers.cc:597] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:42.233722 19203 consensus_peers.cc:597] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:42.238933 19203 consensus_peers.cc:597] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Couldn't send request to peer 6cab1cfd695145f992765d39dbe662c7. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:42.318810 20254 raft_consensus.cc:493] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:42.320417 20254 raft_consensus.cc:515] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:42.322772 20254 leader_election.cc:290] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:42.323530 19203 consensus_peers.cc:597] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.327335 19204 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:42.328563 19203 leader_election.cc:336] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:42.328940 19203 leader_election.cc:304] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:42.329579 20254 raft_consensus.cc:2749] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:42.375339 20254 raft_consensus.cc:493] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:42.375988 20254 raft_consensus.cc:515] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:42.378485 20254 leader_election.cc:290] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:42.379995 19203 consensus_peers.cc:597] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.381810 19203 leader_election.cc:336] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:42.382604 19204 leader_election.cc:336] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.383072 19204 leader_election.cc:304] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:42.383845 20254 raft_consensus.cc:2749] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:42.464346 20254 raft_consensus.cc:493] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:42.465143 20254 raft_consensus.cc:515] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:42.468106 20254 leader_election.cc:290] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:42.472158 19203 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:42.479445 20257 raft_consensus.cc:493] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:42.480048 20257 raft_consensus.cc:515] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
W20260709 12:14:42.481209 19203 leader_election.cc:336] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:42.481765 19203 leader_election.cc:304] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.482347 20257 leader_election.cc:290] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:42.484869 20257 raft_consensus.cc:2749] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:42.488669 20261 raft_consensus.cc:493] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:42.489257 20261 raft_consensus.cc:515] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:42.491631 20261 leader_election.cc:290] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
I20260709 12:14:42.493228 20261 raft_consensus.cc:493] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
W20260709 12:14:42.493858 19203 leader_election.cc:336] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:42.494565 20261 raft_consensus.cc:515] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
W20260709 12:14:42.494771 19204 leader_election.cc:336] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.496986 19204 leader_election.cc:304] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.497570 20261 leader_election.cc:290] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
I20260709 12:14:42.497836 20260 raft_consensus.cc:2749] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:42.498883 19203 leader_election.cc:336] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:42.500689 19204 leader_election.cc:336] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.501161 19204 leader_election.cc:304] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.501958 20260 raft_consensus.cc:2749] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:14:42.502698 19203 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:42.503737 19204 leader_election.cc:336] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.504192 19204 leader_election.cc:304] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:42.505069 20260 raft_consensus.cc:2749] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:42.517786 20260 raft_consensus.cc:493] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.518417 20260 raft_consensus.cc:515] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:42.521031 20260 leader_election.cc:290] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:42.527681 19203 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
W20260709 12:14:42.529155 19204 leader_election.cc:336] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.529663 19204 leader_election.cc:304] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:42.530539 20260 raft_consensus.cc:2749] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:42.543941 20260 raft_consensus.cc:493] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:42.544497 20260 raft_consensus.cc:515] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:42.546548 20260 leader_election.cc:290] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:42.550701 19203 leader_election.cc:336] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:42.551401 19204 leader_election.cc:336] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
I20260709 12:14:42.551754 19204 leader_election.cc:304] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.552393 20260 raft_consensus.cc:2749] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:42.599841 20260 raft_consensus.cc:493] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 666fef4f7e5a4c32813dad8dfade8561)
I20260709 12:14:42.600551 20260 raft_consensus.cc:515] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } } peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } }
I20260709 12:14:42.601523 20261 raft_consensus.cc:493] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:42.602310 20261 raft_consensus.cc:515] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:42.604593 20261 leader_election.cc:290] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
I20260709 12:14:42.606756 20260 leader_election.cc:290] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717), 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851)
W20260709 12:14:42.612464 19204 leader_election.cc:336] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:42.615978 19203 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:42.616686 19203 leader_election.cc:336] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
I20260709 12:14:42.617033 19203 leader_election.cc:304] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 666fef4f7e5a4c32813dad8dfade8561, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.617709 20260 raft_consensus.cc:2749] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:42.618973 19203 leader_election.cc:336] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:42.619469 19203 leader_election.cc:304] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.620097 20260 raft_consensus.cc:2749] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:42.759879 19203 consensus_peers.cc:597] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:42.780071 19203 consensus_peers.cc:597] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:42.804013 20260 raft_consensus.cc:493] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:42.804858 20260 raft_consensus.cc:515] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:42.807768 20260 leader_election.cc:290] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:42.811323  9261 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 19186)
W20260709 12:14:42.812013 19203 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:42.812898 19203 leader_election.cc:336] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:42.813333 19203 leader_election.cc:304] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:42.814252 20260 raft_consensus.cc:2749] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:42.933354 19203 consensus_peers.cc:597] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Couldn't send request to peer 3fcac1aec8df4c599b4c67550ce347e1. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:43.051546 20260 raft_consensus.cc:493] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:43.052116 20260 raft_consensus.cc:515] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:43.054024 20260 leader_election.cc:290] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:43.058202 19203 leader_election.cc:336] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:43.058954 19203 leader_election.cc:336] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:43.059410 19203 leader_election.cc:304] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:43.060184 20260 raft_consensus.cc:2749] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:43.127439 20260 raft_consensus.cc:493] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6cab1cfd695145f992765d39dbe662c7)
I20260709 12:14:43.128250 20260 raft_consensus.cc:515] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:43.130730 20260 leader_election.cc:290] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:43.135422 19203 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:43.136979 19203 leader_election.cc:336] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:43.137498 19203 leader_election.cc:304] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:43.138386 20261 raft_consensus.cc:2749] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:43.142776 19204 consensus_peers.cc:597] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 -> Peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Couldn't send request to peer 666fef4f7e5a4c32813dad8dfade8561. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:43.254272 20260 raft_consensus.cc:493] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.254943 20260 raft_consensus.cc:515] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cab1cfd695145f992765d39dbe662c7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 32851 } } peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } }
I20260709 12:14:43.257238 20260 leader_election.cc:290] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851), 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327)
W20260709 12:14:43.264298 19203 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6cab1cfd695145f992765d39dbe662c7 (127.9.11.67:32851): Network error: Client connection negotiation failed: client connection to 127.9.11.67:32851: connect: Connection refused (error 111)
W20260709 12:14:43.265540 19203 leader_election.cc:336] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:43.266075 19203 leader_election.cc:304] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 6cab1cfd695145f992765d39dbe662c7
I20260709 12:14:43.266884 20260 raft_consensus.cc:2749] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:43.290261  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:14:43.320762 20261 raft_consensus.cc:493] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.321599 20261 raft_consensus.cc:515] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c0b3983ae6c48108f6f3632116cf5a8" member_type: VOTER last_known_addr { host: "127.9.11.68" port: 33373 } } peers { permanent_uuid: "3fcac1aec8df4c599b4c67550ce347e1" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 43327 } } peers { permanent_uuid: "666fef4f7e5a4c32813dad8dfade8561" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34717 } }
I20260709 12:14:43.324280 20261 leader_election.cc:290] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327), 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717)
W20260709 12:14:43.328927 19204 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 666fef4f7e5a4c32813dad8dfade8561 (127.9.11.66:34717): Network error: Client connection negotiation failed: client connection to 127.9.11.66:34717: connect: Connection refused (error 111)
W20260709 12:14:43.330080 19203 leader_election.cc:336] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3fcac1aec8df4c599b4c67550ce347e1 (127.9.11.65:43327): Network error: Client connection negotiation failed: client connection to 127.9.11.65:43327: connect: Connection refused (error 111)
I20260709 12:14:43.330564 19203 leader_election.cc:304] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c0b3983ae6c48108f6f3632116cf5a8; no voters: 3fcac1aec8df4c599b4c67550ce347e1, 666fef4f7e5a4c32813dad8dfade8561
I20260709 12:14:43.331382 20261 raft_consensus.cc:2749] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:43.403832  9261 tablet_server.cc:179] TabletServer@127.9.11.68:0 shutting down...
I20260709 12:14:43.451723  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:43.452702  9261 tablet_replica.cc:333] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.453521  9261 raft_consensus.cc:2243] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.454710  9261 raft_consensus.cc:2272] T 43f821d9236644bcb6676552b6ee4019 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.457752  9261 tablet_replica.cc:333] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.458705  9261 raft_consensus.cc:2243] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.459331  9261 raft_consensus.cc:2272] T ee5a381266c643c9a8b1ea339f87b328 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.461964  9261 tablet_replica.cc:333] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.462595  9261 raft_consensus.cc:2243] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.463029  9261 raft_consensus.cc:2272] T 3cfc76678bd1474fb6f9826360c9e870 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.465152  9261 tablet_replica.cc:333] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.465778  9261 raft_consensus.cc:2243] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.466631  9261 raft_consensus.cc:2272] T 6079f38db31043d2bc846667f08d7d1b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.468760  9261 tablet_replica.cc:333] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.469372  9261 raft_consensus.cc:2243] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.469784  9261 raft_consensus.cc:2272] T c0fdf1bfb7c54f7da0aa45218c041b08 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.471989  9261 tablet_replica.cc:333] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.472604  9261 raft_consensus.cc:2243] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.473016  9261 raft_consensus.cc:2272] T 5a3cd9e4dbb041ddb6775aa9ff22575b P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.475144  9261 tablet_replica.cc:333] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.475713  9261 raft_consensus.cc:2243] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.476559  9261 raft_consensus.cc:2272] T 0ccbbf4c1622458faa667dc368b16eff P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.478628  9261 tablet_replica.cc:333] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.479286  9261 raft_consensus.cc:2243] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.479729  9261 raft_consensus.cc:2272] T bee0ab48d58247cf8e32af4e5a43e66a P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.481801  9261 tablet_replica.cc:333] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.482398  9261 raft_consensus.cc:2243] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.483397  9261 raft_consensus.cc:2272] T 62d133466fbe4df8a6e09156cec6acf4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.485648  9261 tablet_replica.cc:333] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.486286  9261 raft_consensus.cc:2243] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.486692  9261 raft_consensus.cc:2272] T 718f20f68825449c836f67bd5a4bae55 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.488909  9261 tablet_replica.cc:333] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.489537  9261 raft_consensus.cc:2243] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.489952  9261 raft_consensus.cc:2272] T 9061504a72094d2abbe26a00b4998e9c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.492036  9261 tablet_replica.cc:333] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.492645  9261 raft_consensus.cc:2243] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.493043  9261 raft_consensus.cc:2272] T 7a1299cc26f84cbaa6278121975f7961 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.495297  9261 tablet_replica.cc:333] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.495957  9261 raft_consensus.cc:2243] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.496980  9261 raft_consensus.cc:2272] T 9de7d5ebe62f4f1198a4076ef2c81d34 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.499287  9261 tablet_replica.cc:333] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.499883  9261 raft_consensus.cc:2243] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.500770  9261 raft_consensus.cc:2272] T 5ecdce0779954a328062dfca1a0364e4 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.502961  9261 tablet_replica.cc:333] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.503614  9261 raft_consensus.cc:2243] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.504076  9261 raft_consensus.cc:2272] T 5d792b44d6c54e1490e101d60dbebb82 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.506124  9261 tablet_replica.cc:333] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.506707  9261 raft_consensus.cc:2243] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.507680  9261 raft_consensus.cc:2272] T 3d1280ae7e994dc59cf16f8ddef18b72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.509866  9261 tablet_replica.cc:333] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.510600  9261 raft_consensus.cc:2243] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.511121  9261 raft_consensus.cc:2272] T 428f9c2176004b6aa007c6a7af377fcb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.514015  9261 tablet_replica.cc:333] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.514770  9261 raft_consensus.cc:2243] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.515710  9261 raft_consensus.cc:2272] T 3f3e80b40cf94fca9db5cc0ea02e4c48 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.517719  9261 tablet_replica.cc:333] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.518299  9261 raft_consensus.cc:2243] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.518740  9261 raft_consensus.cc:2272] T 24dc0ead3da3436d97931d4aff80a357 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.520828  9261 tablet_replica.cc:333] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.521401  9261 raft_consensus.cc:2243] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.522333  9261 raft_consensus.cc:2272] T 7726c731410a492188ef1b403a54f3bc P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.524623  9261 tablet_replica.cc:333] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.525279  9261 raft_consensus.cc:2243] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.526145  9261 raft_consensus.cc:2272] T 20a717f41e3045c8b600ca072c4b1efd P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.528177  9261 tablet_replica.cc:333] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.528777  9261 raft_consensus.cc:2243] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.529619  9261 raft_consensus.cc:2272] T b4e5cc9ac6c94ccda322218e1f7eb2cb P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.531843  9261 tablet_replica.cc:333] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.532457  9261 raft_consensus.cc:2243] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.532886  9261 raft_consensus.cc:2272] T 67045c0c43a64d4a896e1928eda8289f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.534864  9261 tablet_replica.cc:333] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.535501  9261 raft_consensus.cc:2243] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.536410  9261 raft_consensus.cc:2272] T 43888d54ff1d470a8d63fb846d1d9282 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.538532  9261 tablet_replica.cc:333] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.539147  9261 raft_consensus.cc:2243] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.540019  9261 raft_consensus.cc:2272] T 489f2ef3c1464126b44ded0df97dbee1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.542085  9261 tablet_replica.cc:333] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.542692  9261 raft_consensus.cc:2243] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.543176  9261 raft_consensus.cc:2272] T 8747727903d14849a4d18d8676bbf505 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.545185  9261 tablet_replica.cc:333] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.545763  9261 raft_consensus.cc:2243] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.546566  9261 pending_rounds.cc:70] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: Trying to abort 1 pending ops.
I20260709 12:14:43.546746  9261 pending_rounds.cc:77] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622644244758528 op_type: NO_OP noop_request { }
I20260709 12:14:43.547075  9261 raft_consensus.cc:2889] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.547355  9261 raft_consensus.cc:2272] T 9ec43e6601ae4d8cab4e09c7e77632ae P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.549396  9261 tablet_replica.cc:333] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.550099  9261 raft_consensus.cc:2243] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.550570  9261 raft_consensus.cc:2272] T 9c7af1ca05aa414999a70c06b637cb56 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.552659  9261 tablet_replica.cc:333] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.553205  9261 raft_consensus.cc:2243] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.554198  9261 raft_consensus.cc:2272] T b9af570d255149c5aedd8856d3698e12 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.556574  9261 tablet_replica.cc:333] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.557168  9261 raft_consensus.cc:2243] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.557631  9261 raft_consensus.cc:2272] T 81a0d1ae8dd1431d9f241ba27f35f18f P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.559741  9261 tablet_replica.cc:333] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.560303  9261 raft_consensus.cc:2243] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.561189  9261 pending_rounds.cc:70] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: Trying to abort 1 pending ops.
I20260709 12:14:43.561419  9261 pending_rounds.cc:77] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305622644813557760 op_type: NO_OP noop_request { }
I20260709 12:14:43.561709  9261 raft_consensus.cc:2889] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.562000  9261 raft_consensus.cc:2272] T d99ec37464bd4047a9485e16e72301e9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.564599  9261 tablet_replica.cc:333] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.565265  9261 raft_consensus.cc:2243] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.565696  9261 raft_consensus.cc:2272] T 0741f1cc27bd423d9b9c55f4665cfce1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.568276  9261 tablet_replica.cc:333] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.569005  9261 raft_consensus.cc:2243] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.569500  9261 raft_consensus.cc:2272] T eff579fc38a94049a77efcb07b6bf35c P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.572022  9261 tablet_replica.cc:333] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.572645  9261 raft_consensus.cc:2243] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.573114  9261 raft_consensus.cc:2272] T 5373fb94420c4369bab88209ddbd5324 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.575646  9261 tablet_replica.cc:333] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.576292  9261 raft_consensus.cc:2243] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.576823  9261 raft_consensus.cc:2272] T 169518fbf85c4378af0ff5af5df6dff1 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.579156  9261 tablet_replica.cc:333] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.579742  9261 raft_consensus.cc:2243] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.580837  9261 raft_consensus.cc:2272] T a377e1ad0954438589b509e2491c8a72 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.583071  9261 tablet_replica.cc:333] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.583666  9261 raft_consensus.cc:2243] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.584142  9261 raft_consensus.cc:2272] T c890b118e3b14e4291cb99b7c9f31ae2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.586354  9261 tablet_replica.cc:333] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.587127  9261 raft_consensus.cc:2243] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.587596  9261 raft_consensus.cc:2272] T 0cc2ff8089f3493a8d2e92e4deb28750 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.589763  9261 tablet_replica.cc:333] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.590371  9261 raft_consensus.cc:2243] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.591776  9261 raft_consensus.cc:2272] T e913e5c22bd646e1944cf0bac9dae4c9 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.594247  9261 tablet_replica.cc:333] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.594784  9261 raft_consensus.cc:2243] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.595274  9261 raft_consensus.cc:2272] T d311e304cfc84db5a4dfc30459f4d72e P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.597502  9261 tablet_replica.cc:333] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.598181  9261 raft_consensus.cc:2243] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.598630  9261 raft_consensus.cc:2272] T f9eff763afc14d8aa7bbd4505c26d0c3 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.600853  9261 tablet_replica.cc:333] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.601328  9261 raft_consensus.cc:2243] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.601751  9261 raft_consensus.cc:2272] T 2ff6615d74da465f9105797edb9c3698 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.603997  9261 tablet_replica.cc:333] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.604458  9261 raft_consensus.cc:2243] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.605361  9261 pending_rounds.cc:70] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: Trying to abort 1 pending ops.
I20260709 12:14:43.605553  9261 pending_rounds.cc:77] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305622644936302592 op_type: NO_OP noop_request { }
I20260709 12:14:43.605808  9261 raft_consensus.cc:2889] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.606063  9261 raft_consensus.cc:2272] T fbccd6116cc5452ebe3b6f446963bcd2 P 2c0b3983ae6c48108f6f3632116cf5a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.608361  9261 tablet_replica.cc:333] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8: stopping tablet replica
I20260709 12:14:43.608817  9261 raft_consensus.cc:2243] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.609211  9261 raft_consensus.cc:2272] T 3d0ebf6b04c74f4c994950e78fccf2fe P 2c0b3983ae6c48108f6f3632116cf5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.710264  9261 tablet_server.cc:196] TabletServer@127.9.11.68:0 shutdown complete.
I20260709 12:14:43.793717  9261 master.cc:562] Master@127.9.11.126:34843 shutting down...
I20260709 12:14:43.829577  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.830312  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.830696  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P 01467632324a4240b0c52f8513bb99ba: stopping tablet replica
I20260709 12:14:43.857115  9261 master.cc:584] Master@127.9.11.126:34843 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (39870 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:14:43.963769  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:38165
I20260709 12:14:43.965138  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:43.972291 20264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:43.972417 20263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:43.973474 20266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:43.976104  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:43.977720  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:43.977936  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:43.978061  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599283978049 us; error 0 us; skew 500 ppm
I20260709 12:14:43.978652  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:43.981660  9261 webserver.cc:533] Webserver started at http://127.9.11.126:34583/ using document root <none> and password file <none>
I20260709 12:14:43.982200  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:43.982381  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:43.982628  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:43.983954  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "47f3305b26394df89f9f32e82514e915"
format_stamp: "Formatted at 2026-07-09 12:14:43 on dist-test-slave-k289"
I20260709 12:14:43.988976  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:14:43.992961 20271 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:43.993906  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:14:43.994218  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "47f3305b26394df89f9f32e82514e915"
format_stamp: "Formatted at 2026-07-09 12:14:43 on dist-test-slave-k289"
I20260709 12:14:43.994506  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:44.018730  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:44.020316  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:44.068197  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:38165
I20260709 12:14:44.068337 20322 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:38165 every 8 connection(s)
I20260709 12:14:44.072932 20323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.087481 20323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915: Bootstrap starting.
I20260709 12:14:44.093633 20323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:44.099498 20323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915: No bootstrap required, opened a new log
I20260709 12:14:44.102313 20323 raft_consensus.cc:359] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f3305b26394df89f9f32e82514e915" member_type: VOTER }
I20260709 12:14:44.102809 20323 raft_consensus.cc:385] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:44.103106 20323 raft_consensus.cc:740] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f3305b26394df89f9f32e82514e915, State: Initialized, Role: FOLLOWER
I20260709 12:14:44.103788 20323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f3305b26394df89f9f32e82514e915" member_type: VOTER }
I20260709 12:14:44.104334 20323 raft_consensus.cc:399] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:14:44.104593 20323 raft_consensus.cc:493] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:14:44.104921 20323 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:44.111142 20323 raft_consensus.cc:515] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f3305b26394df89f9f32e82514e915" member_type: VOTER }
I20260709 12:14:44.111814 20323 leader_election.cc:304] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47f3305b26394df89f9f32e82514e915; no voters: 
I20260709 12:14:44.113410 20323 leader_election.cc:290] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:14:44.113874 20326 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:44.115477 20326 raft_consensus.cc:697] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 1 LEADER]: Becoming Leader. State: Replica: 47f3305b26394df89f9f32e82514e915, State: Running, Role: LEADER
I20260709 12:14:44.116278 20326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f3305b26394df89f9f32e82514e915" member_type: VOTER }
I20260709 12:14:44.117278 20323 sys_catalog.cc:565] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:14:44.121294 20327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47f3305b26394df89f9f32e82514e915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f3305b26394df89f9f32e82514e915" member_type: VOTER } }
I20260709 12:14:44.121351 20328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47f3305b26394df89f9f32e82514e915. Latest consensus state: current_term: 1 leader_uuid: "47f3305b26394df89f9f32e82514e915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47f3305b26394df89f9f32e82514e915" member_type: VOTER } }
I20260709 12:14:44.122474 20327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [sys.catalog]: This master's current role is: LEADER
I20260709 12:14:44.122675 20328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [sys.catalog]: This master's current role is: LEADER
I20260709 12:14:44.128424 20332 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:14:44.137441 20332 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:14:44.139493  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:14:44.148376 20332 catalog_manager.cc:1383] Generated new cluster ID: a50de8e36bf343cebab2aa82dc807d9a
I20260709 12:14:44.148782 20332 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:14:44.169030 20332 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:14:44.170605 20332 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:14:44.192551 20332 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915: Generated new TSK 0
I20260709 12:14:44.193372 20332 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:14:44.207742  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:44.215247 20344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:44.216952 20345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:44.219458  9261 server_base.cc:1061] running on GCE node
W20260709 12:14:44.220028 20347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:44.221293  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:44.221501  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:44.221627  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599284221615 us; error 0 us; skew 500 ppm
I20260709 12:14:44.222157  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:44.224795  9261 webserver.cc:533] Webserver started at http://127.9.11.65:35257/ using document root <none> and password file <none>
I20260709 12:14:44.225303  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:44.225469  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:44.225703  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:44.226928  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:44.231923  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:14:44.235769 20352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.236559  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:14:44.236882  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:44.237203  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:44.265820  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:44.267431  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:44.269291  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:44.271953  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:44.272194  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.272457  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:44.272660  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.322551  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:42053
I20260709 12:14:44.322729 20414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:42053 every 8 connection(s)
I20260709 12:14:44.328866  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:44.337237 20419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:44.339358 20420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:44.342077 20422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:44.342466  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:44.344265  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:44.344539  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:44.344750  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599284344710 us; error 0 us; skew 500 ppm
I20260709 12:14:44.345546  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:44.348759  9261 webserver.cc:533] Webserver started at http://127.9.11.66:36665/ using document root <none> and password file <none>
I20260709 12:14:44.349342  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:44.349589  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:44.349910 20415 heartbeater.cc:344] Connected to a master server at 127.9.11.126:38165
I20260709 12:14:44.349977  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:44.350565 20415 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:44.351507 20415 heartbeater.cc:507] Master 127.9.11.126:38165 requested a full tablet report, sending...
I20260709 12:14:44.351665  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:44.354082 20288 ts_manager.cc:194] Registered new tserver with Master: 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053)
I20260709 12:14:44.356317 20288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51466
I20260709 12:14:44.357853  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:14:44.362013 20427 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.362834  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:14:44.363307  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:44.363643  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:44.385097  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:44.386639  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:44.388504  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:44.391047  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:44.391289  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.391543  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:44.391723  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.441812  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:35035
I20260709 12:14:44.441936 20489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:35035 every 8 connection(s)
I20260709 12:14:44.447621  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:44.456259 20493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:44.456667 20494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:44.460083 20496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:44.460421  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:44.461376 20490 heartbeater.cc:344] Connected to a master server at 127.9.11.126:38165
I20260709 12:14:44.461764  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:14:44.461890 20490 heartbeater.cc:461] Registering TS with master...
W20260709 12:14:44.462059  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:44.462328  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599284462310 us; error 0 us; skew 500 ppm
I20260709 12:14:44.462977  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:44.462940 20490 heartbeater.cc:507] Master 127.9.11.126:38165 requested a full tablet report, sending...
I20260709 12:14:44.465816 20288 ts_manager.cc:194] Registered new tserver with Master: 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:44.466286  9261 webserver.cc:533] Webserver started at http://127.9.11.67:35991/ using document root <none> and password file <none>
I20260709 12:14:44.466888  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:44.467149  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:44.467489  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:44.467721 20288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51482
I20260709 12:14:44.468991  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "d022a54d46334c568c045b076b6ceed8"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:44.474542  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:14:44.478438 20501 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.479471  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:14:44.479842  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "d022a54d46334c568c045b076b6ceed8"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:44.480189  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:44.493929  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:44.495473  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:44.497056  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:44.499686  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:44.499943  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.500196  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:44.500393  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:44.548192  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:33443
I20260709 12:14:44.548316 20563 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:33443 every 8 connection(s)
I20260709 12:14:44.564716 20564 heartbeater.cc:344] Connected to a master server at 127.9.11.126:38165
I20260709 12:14:44.565256 20564 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:44.566181 20564 heartbeater.cc:507] Master 127.9.11.126:38165 requested a full tablet report, sending...
I20260709 12:14:44.568377 20288 ts_manager.cc:194] Registered new tserver with Master: d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:44.569383  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017345464s
I20260709 12:14:44.570045 20288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51498
I20260709 12:14:44.599835 20288 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51508:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:14:44.603266 20288 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:14:44.690022 20379 tablet_service.cc:1511] Processing CreateTablet for tablet 6c13381f98574253ba7377725c8391ee (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:14:44.690214 20380 tablet_service.cc:1511] Processing CreateTablet for tablet f18649d815484e82b8c54d7718cecc11 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:14:44.691958 20379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c13381f98574253ba7377725c8391ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.696818 20380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f18649d815484e82b8c54d7718cecc11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.696378 20378 tablet_service.cc:1511] Processing CreateTablet for tablet 043e3ceed12e4dbba399d7e6930749b8 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:14:44.698144 20378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043e3ceed12e4dbba399d7e6930749b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.699851 20377 tablet_service.cc:1511] Processing CreateTablet for tablet 016522e4a22b4ef4969964155f73bae3 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:14:44.701460 20377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 016522e4a22b4ef4969964155f73bae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.704337 20375 tablet_service.cc:1511] Processing CreateTablet for tablet 14c0c3180a764d39bf327bd26b40afb2 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:14:44.705943 20375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c0c3180a764d39bf327bd26b40afb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.707645 20376 tablet_service.cc:1511] Processing CreateTablet for tablet 5a927a0941f34205b9c4c622b45168e4 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:14:44.709367 20376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a927a0941f34205b9c4c622b45168e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.712376 20374 tablet_service.cc:1511] Processing CreateTablet for tablet dff3959e3c6246cd96ee165ee236a06d (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:14:44.714020 20374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dff3959e3c6246cd96ee165ee236a06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.714542 20373 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8cff5054984073936d856d110a8a94 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:14:44.716087 20373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8cff5054984073936d856d110a8a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.717710 20529 tablet_service.cc:1511] Processing CreateTablet for tablet f18649d815484e82b8c54d7718cecc11 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:14:44.719210 20524 tablet_service.cc:1511] Processing CreateTablet for tablet 14c0c3180a764d39bf327bd26b40afb2 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:14:44.720209 20528 tablet_service.cc:1511] Processing CreateTablet for tablet 6c13381f98574253ba7377725c8391ee (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:14:44.721661 20523 tablet_service.cc:1511] Processing CreateTablet for tablet dff3959e3c6246cd96ee165ee236a06d (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:14:44.722795 20527 tablet_service.cc:1511] Processing CreateTablet for tablet 043e3ceed12e4dbba399d7e6930749b8 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:14:44.725365 20525 tablet_service.cc:1511] Processing CreateTablet for tablet 5a927a0941f34205b9c4c622b45168e4 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:14:44.729616 20454 tablet_service.cc:1511] Processing CreateTablet for tablet 6c13381f98574253ba7377725c8391ee (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:14:44.731235 20454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c13381f98574253ba7377725c8391ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.731753 20450 tablet_service.cc:1511] Processing CreateTablet for tablet 14c0c3180a764d39bf327bd26b40afb2 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:14:44.733269 20450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c0c3180a764d39bf327bd26b40afb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.733490 20453 tablet_service.cc:1511] Processing CreateTablet for tablet 043e3ceed12e4dbba399d7e6930749b8 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:14:44.734975 20453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043e3ceed12e4dbba399d7e6930749b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.736334 20455 tablet_service.cc:1511] Processing CreateTablet for tablet f18649d815484e82b8c54d7718cecc11 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:14:44.740092 20529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f18649d815484e82b8c54d7718cecc11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.736341 20452 tablet_service.cc:1511] Processing CreateTablet for tablet 016522e4a22b4ef4969964155f73bae3 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:14:44.740904 20524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c0c3180a764d39bf327bd26b40afb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.737655 20451 tablet_service.cc:1511] Processing CreateTablet for tablet 5a927a0941f34205b9c4c622b45168e4 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:14:44.738338 20449 tablet_service.cc:1511] Processing CreateTablet for tablet dff3959e3c6246cd96ee165ee236a06d (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:14:44.741672 20528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c13381f98574253ba7377725c8391ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.738495 20526 tablet_service.cc:1511] Processing CreateTablet for tablet 016522e4a22b4ef4969964155f73bae3 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:14:44.745453 20523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dff3959e3c6246cd96ee165ee236a06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.746264 20526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 016522e4a22b4ef4969964155f73bae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.747385 20527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043e3ceed12e4dbba399d7e6930749b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.748225 20449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dff3959e3c6246cd96ee165ee236a06d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.748622 20525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a927a0941f34205b9c4c622b45168e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.749619 20451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a927a0941f34205b9c4c622b45168e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.751083 20455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f18649d815484e82b8c54d7718cecc11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.752714 20452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 016522e4a22b4ef4969964155f73bae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.795308 20448 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8cff5054984073936d856d110a8a94 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:14:44.797216 20448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8cff5054984073936d856d110a8a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.816250 20522 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8cff5054984073936d856d110a8a94 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:14:44.818557 20522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8cff5054984073936d856d110a8a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.832828 20375 tablet_service.cc:1511] Processing CreateTablet for tablet b28754f70ce84703a760f5614f45ccf8 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:14:44.834679 20375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b28754f70ce84703a760f5614f45ccf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.847394 20447 tablet_service.cc:1511] Processing CreateTablet for tablet b28754f70ce84703a760f5614f45ccf8 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:14:44.849277 20447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b28754f70ce84703a760f5614f45ccf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.870275 20521 tablet_service.cc:1511] Processing CreateTablet for tablet b28754f70ce84703a760f5614f45ccf8 (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:14:44.872421 20521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b28754f70ce84703a760f5614f45ccf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.899456 20579 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:44.912288 20580 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:44.916666 20446 tablet_service.cc:1511] Processing CreateTablet for tablet 02929838c5ed4c25b31cc52587d022da (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:14:44.918491 20446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02929838c5ed4c25b31cc52587d022da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.923029 20581 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:44.928141 20380 tablet_service.cc:1511] Processing CreateTablet for tablet 02929838c5ed4c25b31cc52587d022da (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:14:44.929858 20380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02929838c5ed4c25b31cc52587d022da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.931243 20581 tablet_bootstrap.cc:654] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:44.943456 20529 tablet_service.cc:1511] Processing CreateTablet for tablet 02929838c5ed4c25b31cc52587d022da (DEFAULT_TABLE table=test-workload [id=43c1f64d91164e3791018e8adefbfb72]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:14:44.946686 20529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02929838c5ed4c25b31cc52587d022da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:44.972898 20581 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:44.976420 20581 ts_tablet_manager.cc:1403] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.054s	user 0.003s	sys 0.015s
I20260709 12:14:44.973731 20580 tablet_bootstrap.cc:654] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:44.972990 20579 tablet_bootstrap.cc:654] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:44.982962 20581 raft_consensus.cc:359] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:44.983937 20581 raft_consensus.cc:385] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:44.984297 20581 raft_consensus.cc:740] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:44.985248 20581 consensus_queue.cc:260] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:44.990912 20490 heartbeater.cc:499] Master 127.9.11.126:38165 was elected leader, sending a full tablet report...
I20260709 12:14:44.991492 20581 ts_tablet_manager.cc:1434] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 12:14:44.992871 20581 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:44.990835 20579 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:44.993544 20580 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:44.994223 20579 ts_tablet_manager.cc:1403] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.095s	user 0.009s	sys 0.008s
I20260709 12:14:44.994287 20580 ts_tablet_manager.cc:1403] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.082s	user 0.016s	sys 0.000s
I20260709 12:14:44.998643 20585 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.999317 20579 raft_consensus.cc:359] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:44.999430 20585 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:44.998325 20580 raft_consensus.cc:359] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.000748 20581 tablet_bootstrap.cc:654] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.001052 20580 raft_consensus.cc:385] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.001534 20580 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.002635 20580 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.003598 20579 raft_consensus.cc:385] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.004069 20579 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.003132 20585 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.005919 20579 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.072165 20580 ts_tablet_manager.cc:1434] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.077s	user 0.018s	sys 0.043s
I20260709 12:14:45.075183 20564 heartbeater.cc:499] Master 127.9.11.126:38165 was elected leader, sending a full tablet report...
I20260709 12:14:45.078634 20579 ts_tablet_manager.cc:1434] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.084s	user 0.030s	sys 0.039s
W20260709 12:14:45.081100 20416 tablet.cc:2406] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:14:45.085117 20581 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.085798 20581 ts_tablet_manager.cc:1403] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.093s	user 0.030s	sys 0.048s
I20260709 12:14:45.086017 20580 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.084646 20390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.089514 20581 raft_consensus.cc:359] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.091284 20581 raft_consensus.cc:385] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.091656 20581 raft_consensus.cc:740] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.090413 20415 heartbeater.cc:499] Master 127.9.11.126:38165 was elected leader, sending a full tablet report...
W20260709 12:14:45.092682 20428 leader_election.cc:343] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.093333 20581 consensus_queue.cc:260] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.095312 20579 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.096238 20581 ts_tablet_manager.cc:1434] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:14:45.097483 20581 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.100060 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.100967 20580 tablet_bootstrap.cc:654] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:45.102480 20430 leader_election.cc:343] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.103169 20579 tablet_bootstrap.cc:654] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.103119 20430 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a; no voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8
I20260709 12:14:45.104233 20585 raft_consensus.cc:2749] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.104848 20581 tablet_bootstrap.cc:654] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:45.167955 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:45.171281 20580 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.173439 20580 ts_tablet_manager.cc:1403] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.088s	user 0.018s	sys 0.006s
I20260709 12:14:45.174333 20579 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.175902 20579 ts_tablet_manager.cc:1403] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.012s
I20260709 12:14:45.181790 20581 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.182099 20580 raft_consensus.cc:359] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.182974 20579 raft_consensus.cc:359] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.183785 20581 ts_tablet_manager.cc:1403] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.005s
I20260709 12:14:45.184320 20580 raft_consensus.cc:385] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.184374 20579 raft_consensus.cc:385] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.184947 20580 raft_consensus.cc:740] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.185047 20579 raft_consensus.cc:740] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.186704 20579 consensus_queue.cc:260] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.186683 20580 consensus_queue.cc:260] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.190762 20579 ts_tablet_manager.cc:1434] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 12:14:45.191790 20581 raft_consensus.cc:359] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.193432 20579 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.193742 20581 raft_consensus.cc:385] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.194278 20581 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.191804 20580 ts_tablet_manager.cc:1434] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.009s
I20260709 12:14:45.195680 20581 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.199182 20581 ts_tablet_manager.cc:1434] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.005s
I20260709 12:14:45.200906 20580 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.201716 20581 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.209203 20579 tablet_bootstrap.cc:654] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.210843 20580 tablet_bootstrap.cc:654] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.217571 20581 tablet_bootstrap.cc:654] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.228808 20579 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.229255 20581 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.229468 20579 ts_tablet_manager.cc:1403] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.011s
I20260709 12:14:45.230072 20581 ts_tablet_manager.cc:1403] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.010s
I20260709 12:14:45.231632 20580 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.232285 20580 ts_tablet_manager.cc:1403] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.014s
I20260709 12:14:45.233763 20579 raft_consensus.cc:359] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.234092 20581 raft_consensus.cc:359] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.234704 20579 raft_consensus.cc:385] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.234972 20581 raft_consensus.cc:385] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.235141 20579 raft_consensus.cc:740] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.235419 20581 raft_consensus.cc:740] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.236438 20581 consensus_queue.cc:260] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.236439 20580 raft_consensus.cc:359] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.236446 20579 consensus_queue.cc:260] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.237572 20580 raft_consensus.cc:385] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.237944 20580 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.239108 20581 ts_tablet_manager.cc:1434] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:14:45.240125 20579 ts_tablet_manager.cc:1434] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:14:45.241055 20581 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.242132 20579 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.239104 20580 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.250160 20580 ts_tablet_manager.cc:1434] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:14:45.251029 20581 tablet_bootstrap.cc:654] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.251890 20579 tablet_bootstrap.cc:654] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.252203 20580 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.255843 20587 raft_consensus.cc:493] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.256665 20587 raft_consensus.cc:515] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.263906 20587 leader_election.cc:290] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.283826 20580 tablet_bootstrap.cc:654] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.300482 20580 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.301326 20580 ts_tablet_manager.cc:1403] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.004s
I20260709 12:14:45.305330 20390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
W20260709 12:14:45.308233 20502 leader_election.cc:343] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.305303 20579 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.308476 20580 raft_consensus.cc:359] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.309118 20581 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.309631 20580 raft_consensus.cc:385] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.310003 20581 ts_tablet_manager.cc:1403] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.070s	user 0.028s	sys 0.011s
I20260709 12:14:45.310096 20580 raft_consensus.cc:740] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.309671 20579 ts_tablet_manager.cc:1403] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.068s	user 0.031s	sys 0.021s
I20260709 12:14:45.312085 20580 consensus_queue.cc:260] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.314201 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.314452 20581 raft_consensus.cc:359] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.315392 20581 raft_consensus.cc:385] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.314915 20579 raft_consensus.cc:359] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.315723 20581 raft_consensus.cc:740] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.315847 20579 raft_consensus.cc:385] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.316679 20579 raft_consensus.cc:740] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
W20260709 12:14:45.316962 20504 leader_election.cc:343] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.317062 20581 consensus_queue.cc:260] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.317729 20504 leader_election.cc:304] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:45.319224 20587 raft_consensus.cc:2749] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.317616 20579 consensus_queue.cc:260] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.320696 20580 ts_tablet_manager.cc:1434] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:14:45.322497 20580 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.322574 20579 ts_tablet_manager.cc:1434] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.006s
I20260709 12:14:45.325256 20581 ts_tablet_manager.cc:1434] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:14:45.327307 20581 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.329887 20579 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.330426 20587 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.331523 20587 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.332414 20580 tablet_bootstrap.cc:654] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.337464 20581 tablet_bootstrap.cc:654] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.338701 20587 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.340195 20390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.342268 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
W20260709 12:14:45.343153 20502 leader_election.cc:343] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:14:45.344633 20504 leader_election.cc:343] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.345259 20504 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:45.346307 20587 raft_consensus.cc:2749] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.351771 20579 tablet_bootstrap.cc:654] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.366973 20579 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.367862 20579 ts_tablet_manager.cc:1403] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.009s
I20260709 12:14:45.372148 20581 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.372128 20579 raft_consensus.cc:359] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.372903 20581 ts_tablet_manager.cc:1403] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.022s
I20260709 12:14:45.373121 20579 raft_consensus.cc:385] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.373423 20580 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.373548 20579 raft_consensus.cc:740] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.374306 20580 ts_tablet_manager.cc:1403] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.018s
I20260709 12:14:45.374799 20579 consensus_queue.cc:260] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.376910 20581 raft_consensus.cc:359] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.377810 20579 ts_tablet_manager.cc:1434] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:14:45.377856 20581 raft_consensus.cc:385] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.378607 20581 raft_consensus.cc:740] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.378283 20580 raft_consensus.cc:359] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.379359 20580 raft_consensus.cc:385] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.379798 20579 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.379776 20581 consensus_queue.cc:260] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.379808 20580 raft_consensus.cc:740] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.382750 20581 ts_tablet_manager.cc:1434] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.002s
I20260709 12:14:45.382858 20580 consensus_queue.cc:260] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.384460 20581 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.385916 20580 ts_tablet_manager.cc:1434] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:14:45.387614 20580 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.388273 20590 raft_consensus.cc:493] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.389227 20590 raft_consensus.cc:515] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.393548 20581 tablet_bootstrap.cc:654] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.397991 20579 tablet_bootstrap.cc:654] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.399439 20590 leader_election.cc:290] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.403223 20585 raft_consensus.cc:493] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.404111 20585 raft_consensus.cc:515] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.409420 20390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.410315 20390 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.411769 20580 tablet_bootstrap.cc:654] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.413021 20428 leader_election.cc:304] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.410221 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.419464 20539 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.410725 20585 leader_election.cc:290] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.426344 20585 raft_consensus.cc:2804] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.427225 20585 raft_consensus.cc:493] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.428831 20585 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.433020 20579 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.433157 20587 raft_consensus.cc:493] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.434082 20581 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.434168 20579 ts_tablet_manager.cc:1403] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.013s
I20260709 12:14:45.434823 20581 ts_tablet_manager.cc:1403] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.011s
I20260709 12:14:45.434101 20587 raft_consensus.cc:515] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.438691 20579 raft_consensus.cc:359] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.438674 20390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "016522e4a22b4ef4969964155f73bae3" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.439783 20579 raft_consensus.cc:385] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.439713 20581 raft_consensus.cc:359] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.440624 20579 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.442637 20585 raft_consensus.cc:515] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.443120 20581 raft_consensus.cc:385] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.445572 20609 raft_consensus.cc:493] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.446028 20581 raft_consensus.cc:740] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.442880 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "016522e4a22b4ef4969964155f73bae3" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.446219 20609 raft_consensus.cc:515] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.447014 20465 raft_consensus.cc:2468] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 0.
I20260709 12:14:45.447137 20581 consensus_queue.cc:260] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.443554 20587 leader_election.cc:290] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.440069 20390 raft_consensus.cc:2468] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 0.
I20260709 12:14:45.443678 20579 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.449410 20609 leader_election.cc:290] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.450256 20502 leader_election.cc:304] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:45.450896 20579 ts_tablet_manager.cc:1434] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:14:45.451725 20585 leader_election.cc:290] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.459013 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.463604 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.465116 20390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.471971 20390 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.473987 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.475132 20389 raft_consensus.cc:2468] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.475383 20539 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.476135 20585 raft_consensus.cc:493] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.476850 20585 raft_consensus.cc:515] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.477144 20428 leader_election.cc:304] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.478488 20609 raft_consensus.cc:2804] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.478974 20609 raft_consensus.cc:493] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.479386 20609 raft_consensus.cc:3060] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.468662 20587 raft_consensus.cc:2804] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.480242 20587 raft_consensus.cc:493] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.480700 20587 raft_consensus.cc:3060] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.481635 20585 leader_election.cc:290] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.482492 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.483661 20388 raft_consensus.cc:2468] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.485787 20428 leader_election.cc:304] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
W20260709 12:14:45.488773 20430 leader_election.cc:343] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.494895 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.496501 20585 raft_consensus.cc:2804] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.497154 20585 raft_consensus.cc:493] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.498152 20585 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:14:45.501219 20430 leader_election.cc:343] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.510505 20585 raft_consensus.cc:515] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.518549 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.519477 20389 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.525381 20579 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.535490 20389 raft_consensus.cc:2468] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.538889 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.541440 20585 leader_election.cc:290] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.542263 20580 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.544102 20580 ts_tablet_manager.cc:1403] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.157s	user 0.070s	sys 0.036s
W20260709 12:14:45.544675 20430 leader_election.cc:343] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.546782 20428 leader_election.cc:304] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: d022a54d46334c568c045b076b6ceed8
W20260709 12:14:45.547544 20502 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:42053, user_credentials={real_user=slave}} blocked reactor thread for 97401.6us
I20260709 12:14:45.548925 20580 raft_consensus.cc:359] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.549968 20580 raft_consensus.cc:385] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.550474 20580 raft_consensus.cc:740] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.550899 20585 raft_consensus.cc:2804] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.551693 20585 raft_consensus.cc:697] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:45.551815 20580 consensus_queue.cc:260] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.553125 20539 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.553242 20585 consensus_queue.cc:237] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.556317 20590 raft_consensus.cc:493] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.565374 20590 raft_consensus.cc:515] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.567886 20580 ts_tablet_manager.cc:1434] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 12:14:45.568252 20590 leader_election.cc:290] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.557523 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.569334 20538 raft_consensus.cc:2393] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in current term 1: Already voted for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in this term.
I20260709 12:14:45.570230 20587 raft_consensus.cc:515] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.571730 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.572067 20430 leader_election.cc:304] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:45.572573 20539 raft_consensus.cc:2468] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in term 0.
I20260709 12:14:45.572819 20587 leader_election.cc:290] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.575008 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "016522e4a22b4ef4969964155f73bae3" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.575248 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "016522e4a22b4ef4969964155f73bae3" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:45.575769 20389 raft_consensus.cc:3060] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.575966 20465 raft_consensus.cc:3060] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.575939 20579 tablet_bootstrap.cc:654] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.582720 20355 leader_election.cc:304] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:45.589478 20590 raft_consensus.cc:2804] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.589932 20590 raft_consensus.cc:493] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.590325 20590 raft_consensus.cc:3060] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.592890 20579 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.593660 20465 raft_consensus.cc:2468] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 1.
I20260709 12:14:45.561646 20581 ts_tablet_manager.cc:1434] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.126s	user 0.018s	sys 0.000s
I20260709 12:14:45.597930 20581 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.587445 20614 raft_consensus.cc:2804] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.603600 20614 raft_consensus.cc:697] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:45.603717 20464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.604050 20590 raft_consensus.cc:515] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.605002 20614 consensus_queue.cc:237] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:45.604672 20288 catalog_manager.cc:5697] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 1 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.605674 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.606458 20463 raft_consensus.cc:2468] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in term 0.
I20260709 12:14:45.607892 20355 leader_election.cc:304] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8
I20260709 12:14:45.608776 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:45.609539 20463 raft_consensus.cc:3060] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.593818 20579 ts_tablet_manager.cc:1403] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.069s	user 0.024s	sys 0.000s
I20260709 12:14:45.612843 20581 tablet_bootstrap.cc:654] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.560783 20609 raft_consensus.cc:515] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.615289 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.616052 20504 leader_election.cc:304] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:45.617100 20587 raft_consensus.cc:493] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.617177 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.618068 20388 raft_consensus.cc:3060] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.618233 20613 raft_consensus.cc:2804] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.592788 20580 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.588424 20389 raft_consensus.cc:2468] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 1.
I20260709 12:14:45.617707 20587 raft_consensus.cc:515] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.621791 20609 leader_election.cc:290] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.616103 20539 raft_consensus.cc:3060] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.624231 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.624989 20538 raft_consensus.cc:3060] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.627730 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.621886 20587 leader_election.cc:290] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.558921 20390 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.634429 20579 raft_consensus.cc:359] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.635646 20579 raft_consensus.cc:385] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.636067 20579 raft_consensus.cc:740] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.636881 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.643131 20590 leader_election.cc:290] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 election: Requested vote from peers 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.643666 20465 raft_consensus.cc:2393] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d022a54d46334c568c045b076b6ceed8 in current term 1: Already voted for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in this term.
I20260709 12:14:45.643846 20579 consensus_queue.cc:260] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.648177 20580 tablet_bootstrap.cc:654] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.642693 20463 raft_consensus.cc:2468] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in term 1.
I20260709 12:14:45.650421 20538 raft_consensus.cc:2468] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.663309 20581 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.653631 20539 raft_consensus.cc:2468] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in term 1.
I20260709 12:14:45.651357 20617 raft_consensus.cc:2749] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.665835 20581 ts_tablet_manager.cc:1403] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.023s
I20260709 12:14:45.650728 20388 raft_consensus.cc:2468] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.667166 20355 leader_election.cc:304] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.668244 20428 leader_election.cc:304] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.668840 20590 raft_consensus.cc:2804] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.670630 20614 raft_consensus.cc:2804] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.671389 20614 raft_consensus.cc:697] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:45.672734 20613 raft_consensus.cc:697] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 1 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:45.671657 20504 leader_election.cc:304] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:45.673808 20613 consensus_queue.cc:237] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.672746 20614 consensus_queue.cc:237] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.674952 20620 raft_consensus.cc:2749] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.670045 20581 raft_consensus.cc:359] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.679761 20581 raft_consensus.cc:385] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.680166 20581 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.681730 20581 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.682492 20579 ts_tablet_manager.cc:1434] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.070s	user 0.012s	sys 0.007s
I20260709 12:14:45.685717 20579 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.686748 20581 ts_tablet_manager.cc:1434] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:14:45.690577 20581 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.696514 20580 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.697296 20580 ts_tablet_manager.cc:1403] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.105s	user 0.015s	sys 0.007s
I20260709 12:14:45.697425 20590 raft_consensus.cc:697] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 LEADER]: Becoming Leader. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Running, Role: LEADER
I20260709 12:14:45.698751 20590 consensus_queue.cc:237] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.696697 20579 tablet_bootstrap.cc:654] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.701211 20580 raft_consensus.cc:359] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.702164 20580 raft_consensus.cc:385] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.701552 20285 catalog_manager.cc:5697] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:45.702590 20580 raft_consensus.cc:740] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.709833 20580 consensus_queue.cc:260] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.713939 20580 ts_tablet_manager.cc:1434] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:14:45.716137 20580 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.718182 20581 tablet_bootstrap.cc:654] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.732542 20287 catalog_manager.cc:5697] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.756064 20580 tablet_bootstrap.cc:654] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.762748 20288 catalog_manager.cc:5697] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 1 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:45.766821 20288 catalog_manager.cc:5697] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 1 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.770375 20579 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.775593 20579 ts_tablet_manager.cc:1403] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.090s	user 0.026s	sys 0.042s
I20260709 12:14:45.776473 20580 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.777257 20580 ts_tablet_manager.cc:1403] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.020s
I20260709 12:14:45.780126 20579 raft_consensus.cc:359] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.780643 20580 raft_consensus.cc:359] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.781317 20579 raft_consensus.cc:385] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.781607 20580 raft_consensus.cc:385] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.781774 20579 raft_consensus.cc:740] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.783449 20580 raft_consensus.cc:740] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.784230 20579 consensus_queue.cc:260] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.784570 20580 consensus_queue.cc:260] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.785594 20581 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.786258 20581 ts_tablet_manager.cc:1403] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.096s	user 0.021s	sys 0.044s
I20260709 12:14:45.787232 20580 ts_tablet_manager.cc:1434] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:14:45.788646 20580 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.789774 20581 raft_consensus.cc:359] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.790567 20581 raft_consensus.cc:385] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.790884 20581 raft_consensus.cc:740] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.791102 20579 ts_tablet_manager.cc:1434] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 12:14:45.791797 20581 consensus_queue.cc:260] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.792567 20579 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.792809 20614 raft_consensus.cc:493] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.793490 20614 raft_consensus.cc:515] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.794003 20609 raft_consensus.cc:493] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.794245 20581 ts_tablet_manager.cc:1434] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:14:45.794625 20609 raft_consensus.cc:515] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.795557 20581 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a: Bootstrap starting.
I20260709 12:14:45.800904 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28754f70ce84703a760f5614f45ccf8" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.801369 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28754f70ce84703a760f5614f45ccf8" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.803673 20581 tablet_bootstrap.cc:654] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.800906 20580 tablet_bootstrap.cc:654] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
W20260709 12:14:45.803931 20430 leader_election.cc:343] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.804785 20609 leader_election.cc:290] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.805250 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.806138 20388 raft_consensus.cc:2468] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
W20260709 12:14:45.807763 20428 leader_election.cc:343] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:14:45.812728 20613 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.816859 20613 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.819130 20579 tablet_bootstrap.cc:654] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.820533 20581 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a: No bootstrap required, opened a new log
I20260709 12:14:45.821123 20581 ts_tablet_manager.cc:1403] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.007s
I20260709 12:14:45.822001 20613 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.826864 20580 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.827049 20614 leader_election.cc:290] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.826587 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.824803 20428 leader_election.cc:304] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a; no voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8
I20260709 12:14:45.827966 20465 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 0.
I20260709 12:14:45.828275 20590 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.829043 20580 ts_tablet_manager.cc:1403] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.002s
I20260709 12:14:45.828852 20590 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.829501 20504 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:45.830221 20428 leader_election.cc:304] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.830847 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.831179 20609 raft_consensus.cc:2749] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.831621 20538 raft_consensus.cc:2468] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.831969 20609 raft_consensus.cc:2804] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.832435 20609 raft_consensus.cc:493] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.832789 20609 raft_consensus.cc:3060] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.833307 20580 raft_consensus.cc:359] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.833746 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.834122 20580 raft_consensus.cc:385] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.834483 20580 raft_consensus.cc:740] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.834561 20388 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 0.
I20260709 12:14:45.835446 20580 consensus_queue.cc:260] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.837226 20590 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.838616 20580 ts_tablet_manager.cc:1434] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:14:45.840107 20580 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8: Bootstrap starting.
I20260709 12:14:45.840319 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.841029 20465 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in term 0.
I20260709 12:14:45.842592 20355 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.844619 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.836338 20613 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.846328 20613 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.846881 20613 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.825001 20581 raft_consensus.cc:359] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.848266 20581 raft_consensus.cc:385] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.848237 20590 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.849031 20590 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.849499 20590 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.849977 20580 tablet_bootstrap.cc:654] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.851236 20581 raft_consensus.cc:740] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.859035 20581 consensus_queue.cc:260] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.860154 20613 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.861963 20538 raft_consensus.cc:2393] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in current term 1: Already voted for candidate d022a54d46334c568c045b076b6ceed8 in this term.
I20260709 12:14:45.862548 20581 ts_tablet_manager.cc:1434] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.002s
I20260709 12:14:45.854292 20609 raft_consensus.cc:515] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.867349 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.868036 20388 raft_consensus.cc:3060] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.869287 20609 leader_election.cc:290] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.871853 20579 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.872471 20579 ts_tablet_manager.cc:1403] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.004s
I20260709 12:14:45.874365 20590 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.875432 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.876111 20579 raft_consensus.cc:359] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.876933 20579 raft_consensus.cc:385] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.877244 20579 raft_consensus.cc:740] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.877382 20620 raft_consensus.cc:493] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.878060 20579 consensus_queue.cc:260] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.878116 20620 raft_consensus.cc:515] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.876120 20538 raft_consensus.cc:3060] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.880649 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:45.881306 20465 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.882256 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.883158 20389 raft_consensus.cc:2393] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d022a54d46334c568c045b076b6ceed8 in current term 1: Already voted for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in this term.
I20260709 12:14:45.884372 20579 ts_tablet_manager.cc:1434] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:14:45.885727 20579 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:45.885661 20620 leader_election.cc:290] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.886955 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:45.888345 20580 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8: No bootstrap required, opened a new log
I20260709 12:14:45.888978 20580 ts_tablet_manager.cc:1403] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.010s
I20260709 12:14:45.888833 20504 leader_election.cc:304] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:45.890239 20620 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.892527 20580 raft_consensus.cc:359] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.893304 20580 raft_consensus.cc:385] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.893608 20580 raft_consensus.cc:740] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.893926 20579 tablet_bootstrap.cc:654] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:45.894543 20580 consensus_queue.cc:260] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.897257 20580 ts_tablet_manager.cc:1434] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 12:14:45.899119 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.899456 20590 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 election: Requested vote from peers 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.899978 20539 raft_consensus.cc:2393] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in current term 1: Already voted for candidate d022a54d46334c568c045b076b6ceed8 in this term.
I20260709 12:14:45.903370 20465 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in term 1.
I20260709 12:14:45.905927 20355 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: d022a54d46334c568c045b076b6ceed8
I20260709 12:14:45.906891 20590 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.908008 20590 raft_consensus.cc:697] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 LEADER]: Becoming Leader. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Running, Role: LEADER
I20260709 12:14:45.909193 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.909293 20590 consensus_queue.cc:237] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.915552 20613 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:45.915654 20609 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.916473 20609 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.921051 20389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.921833 20389 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.923164 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.923476 20428 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.923961 20539 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.924615 20614 raft_consensus.cc:2804] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.923811 20609 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.925102 20614 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.923597 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:45.925558 20614 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.906972 20388 raft_consensus.cc:2468] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.926290 20465 raft_consensus.cc:2393] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d022a54d46334c568c045b076b6ceed8 in current term 1: Already voted for candidate 9142a6f9cf1742d6a7e2e5f53ef197a4 in this term.
I20260709 12:14:45.927356 20428 leader_election.cc:304] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.928136 20504 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:45.929744 20620 raft_consensus.cc:2749] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.929997 20609 raft_consensus.cc:2804] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.930765 20613 raft_consensus.cc:2749] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:45.930709 20538 raft_consensus.cc:2468] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.932359 20609 raft_consensus.cc:697] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:45.934206 20609 consensus_queue.cc:237] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.936245 20614 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.937027 20579 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: No bootstrap required, opened a new log
I20260709 12:14:45.937659 20579 ts_tablet_manager.cc:1403] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.052s	user 0.034s	sys 0.015s
I20260709 12:14:45.939975 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.940692 20388 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.941957 20579 raft_consensus.cc:359] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.942960 20579 raft_consensus.cc:385] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:45.943336 20579 raft_consensus.cc:740] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:45.944283 20579 consensus_queue.cc:260] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.947130 20579 ts_tablet_manager.cc:1434] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.005s
I20260709 12:14:45.949944 20614 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.950083 20388 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.950587 20287 catalog_manager.cc:5697] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.951643 20428 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.951876 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.952590 20538 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.952781 20614 raft_consensus.cc:2804] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.953680 20614 raft_consensus.cc:697] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:45.954036 20288 catalog_manager.cc:5697] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 1 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.954696 20614 consensus_queue.cc:237] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.961005 20609 raft_consensus.cc:493] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.961705 20609 raft_consensus.cc:515] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.964205 20609 leader_election.cc:290] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.967253 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02929838c5ed4c25b31cc52587d022da" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:45.968096 20539 raft_consensus.cc:2468] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.969491 20430 leader_election.cc:304] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:45.970719 20538 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.971303 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02929838c5ed4c25b31cc52587d022da" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:45.973007 20388 raft_consensus.cc:2468] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 0.
I20260709 12:14:45.974311 20614 raft_consensus.cc:2804] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:45.974790 20614 raft_consensus.cc:493] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.975234 20614 raft_consensus.cc:3060] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.978464 20287 catalog_manager.cc:5697] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 1 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.984218 20614 raft_consensus.cc:515] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:45.986696 20614 leader_election.cc:290] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:45.987258 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02929838c5ed4c25b31cc52587d022da" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:45.987423 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02929838c5ed4c25b31cc52587d022da" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:45.988080 20388 raft_consensus.cc:3060] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.988161 20538 raft_consensus.cc:3060] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:45.994441 20388 raft_consensus.cc:2468] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.995071 20538 raft_consensus.cc:2468] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:45.995725 20428 leader_election.cc:304] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:45.996707 20614 raft_consensus.cc:2804] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:45.997177 20614 raft_consensus.cc:697] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:45.998055 20614 consensus_queue.cc:237] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:46.007628 20287 catalog_manager.cc:5697] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 1 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:46.093680 20609 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.097126 20609 consensus_queue.cc:1048] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:46.099272 20620 consensus_queue.cc:1048] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.110321 20590 consensus_queue.cc:1048] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.121824 20609 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.139791 20587 consensus_queue.cc:1048] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.164036 20623 consensus_queue.cc:1048] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.169237 20614 consensus_queue.cc:1048] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:46.179919 20609 consensus_queue.cc:1048] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:14:46.186554 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:14:46.202476 20614 consensus_queue.cc:1048] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.230682 20613 raft_consensus.cc:493] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.231297 20613 raft_consensus.cc:515] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:46.233556 20613 leader_election.cc:290] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:46.234280 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28754f70ce84703a760f5614f45ccf8" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" is_pre_election: true
I20260709 12:14:46.234728 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28754f70ce84703a760f5614f45ccf8" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:46.235213 20388 raft_consensus.cc:2468] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 0.
I20260709 12:14:46.235489 20463 raft_consensus.cc:2468] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 0.
I20260709 12:14:46.236640 20504 leader_election.cc:304] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:46.237406 20613 raft_consensus.cc:2804] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:46.237742 20613 raft_consensus.cc:493] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.238050 20613 raft_consensus.cc:3060] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:46.244539 20613 raft_consensus.cc:515] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:46.246814 20613 leader_election.cc:290] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:46.247841 20388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28754f70ce84703a760f5614f45ccf8" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:46.248080 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b28754f70ce84703a760f5614f45ccf8" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:46.248585 20388 raft_consensus.cc:3060] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:46.248713 20463 raft_consensus.cc:3060] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:46.255163 20388 raft_consensus.cc:2468] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 1.
I20260709 12:14:46.255173 20463 raft_consensus.cc:2468] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 1.
I20260709 12:14:46.256558 20502 leader_election.cc:304] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:46.257458 20613 raft_consensus.cc:2804] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:46.257958 20613 raft_consensus.cc:697] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 1 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:46.258692 20613 consensus_queue.cc:237] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:46.268821 20287 catalog_manager.cc:5697] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:46.300848 20590 consensus_queue.cc:1048] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:46.314309 20590 consensus_queue.cc:1048] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:46.429924 20614 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:46.443022 20649 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.470001 20649 consensus_queue.cc:1048] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:46.471151 20657 consensus_queue.cc:1048] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.544521 20649 consensus_queue.cc:1048] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.556493 20649 consensus_queue.cc:1048] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:46.570207  9261 ts_manager.cc:295] Set tserver state for 9142a6f9cf1742d6a7e2e5f53ef197a4 to MAINTENANCE_MODE
I20260709 12:14:46.571743  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
I20260709 12:14:46.618429  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:46.619436  9261 tablet_replica.cc:333] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.620525  9261 raft_consensus.cc:2243] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.621044  9261 raft_consensus.cc:2272] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.624310  9261 tablet_replica.cc:333] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.624979  9261 raft_consensus.cc:2243] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:46.625955  9261 raft_consensus.cc:2272] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.628508  9261 tablet_replica.cc:333] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.629140  9261 raft_consensus.cc:2243] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.629575  9261 raft_consensus.cc:2272] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.632629  9261 tablet_replica.cc:333] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.633658  9261 raft_consensus.cc:2243] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.634243  9261 raft_consensus.cc:2272] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:46.634323 20428 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 185 similar messages]
W20260709 12:14:46.638942 20428 consensus_peers.cc:597] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.639151  9261 tablet_replica.cc:333] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.640242  9261 raft_consensus.cc:2243] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.640852  9261 raft_consensus.cc:2272] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.643757  9261 tablet_replica.cc:333] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.644421  9261 raft_consensus.cc:2243] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:46.645458  9261 raft_consensus.cc:2272] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.648490  9261 tablet_replica.cc:333] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.649205  9261 raft_consensus.cc:2243] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.649739  9261 raft_consensus.cc:2272] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.653710  9261 tablet_replica.cc:333] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.654671  9261 raft_consensus.cc:2243] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:46.655066 20502 consensus_peers.cc:597] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.655321  9261 raft_consensus.cc:2272] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:46.657387 20502 consensus_peers.cc:597] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.659526  9261 tablet_replica.cc:333] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.660521  9261 raft_consensus.cc:2243] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.661088  9261 raft_consensus.cc:2272] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.663970  9261 tablet_replica.cc:333] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: stopping tablet replica
I20260709 12:14:46.664690  9261 raft_consensus.cc:2243] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:46.665306  9261 raft_consensus.cc:2272] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:46.705211  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
W20260709 12:14:46.707168 20428 consensus_peers.cc:597] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.755697 20613 consensus_queue.cc:1048] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:14:46.799373 20428 consensus_peers.cc:597] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:46.939926 20428 consensus_peers.cc:597] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.057106 20428 consensus_peers.cc:597] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.069595 20428 consensus_peers.cc:597] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.194352 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:47.668970 20613 raft_consensus.cc:493] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:47.669660 20613 raft_consensus.cc:515] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:47.671938 20613 leader_election.cc:290] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:47.673098 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
W20260709 12:14:47.676141 20502 leader_election.cc:336] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:47.676596 20502 leader_election.cc:304] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:47.677289 20613 raft_consensus.cc:2749] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:14:47.812803 20613 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:47.813505 20613 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:47.816675 20613 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:47.818851 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" is_pre_election: true
I20260709 12:14:47.819252 20649 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:47.820588 20649 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
W20260709 12:14:47.821791 20502 leader_election.cc:336] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:47.821898 20463 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 1.
I20260709 12:14:47.824810 20504 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:47.824883 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:47.825714 20538 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:47.826146 20649 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:47.827144 20613 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:47.827208 20430 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:47.828248 20649 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:47.828686 20649 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:47.829113 20649 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:47.829295 20613 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:47.829802 20613 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:47.830443 20428 leader_election.cc:336] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:47.838764 20649 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:47.838768 20613 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:47.841197 20613 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:47.841295 20649 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:47.842644 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:47.843042 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:47.843732 20463 raft_consensus.cc:2393] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d022a54d46334c568c045b076b6ceed8 in current term 2: Already voted for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in this term.
I20260709 12:14:47.843936 20538 raft_consensus.cc:2393] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4987bb792cfe49b8bdc19fe144b5ec1a in current term 2: Already voted for candidate d022a54d46334c568c045b076b6ceed8 in this term.
W20260709 12:14:47.846401 20502 leader_election.cc:336] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:47.846884 20502 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d022a54d46334c568c045b076b6ceed8; no voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:47.847761 20613 raft_consensus.cc:2749] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:14:47.848392 20428 leader_election.cc:336] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:47.848812 20428 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a; no voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8
I20260709 12:14:47.849421 20649 raft_consensus.cc:2749] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:14:48.086719 20649 raft_consensus.cc:493] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:48.087509 20649 raft_consensus.cc:515] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:48.090382 20649 leader_election.cc:290] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:48.091683 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:48.092626 20538 raft_consensus.cc:2468] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 1.
I20260709 12:14:48.093950 20430 leader_election.cc:304] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
W20260709 12:14:48.094888 20428 leader_election.cc:336] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:48.095046 20649 raft_consensus.cc:2804] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:14:48.095574 20649 raft_consensus.cc:493] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9142a6f9cf1742d6a7e2e5f53ef197a4)
I20260709 12:14:48.096032 20649 raft_consensus.cc:3060] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:48.104290 20649 raft_consensus.cc:515] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:48.106331 20649 leader_election.cc:290] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:48.107643 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "043e3ceed12e4dbba399d7e6930749b8" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:48.108302 20538 raft_consensus.cc:3060] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:14:48.111238 20428 leader_election.cc:336] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:48.116588 20538 raft_consensus.cc:2468] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 2.
I20260709 12:14:48.118108 20430 leader_election.cc:304] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:48.119614 20649 raft_consensus.cc:2804] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:48.120942 20649 raft_consensus.cc:697] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:48.122056 20649 consensus_queue.cc:237] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:48.134455 20285 catalog_manager.cc:5697] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 1 to 2, leader changed from 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65) to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 2 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:48.195523 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:48.642078 20539 raft_consensus.cc:1275] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:48.643947 20649 consensus_queue.cc:1048] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:14:48.667424 20428 consensus_peers.cc:597] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:48.982926 20502 consensus_peers.cc:597] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:48.985260 20502 consensus_peers.cc:597] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:48.985285 20428 consensus_peers.cc:597] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.155465 20428 consensus_peers.cc:597] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.196828 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:14:49.372025 20428 consensus_peers.cc:597] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.395999 20428 consensus_peers.cc:597] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.462517 20428 consensus_peers.cc:597] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:49.518893 20673 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.519580 20673 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:49.521965 20673 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:49.523458 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8" is_pre_election: true
I20260709 12:14:49.524343 20539 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 2.
I20260709 12:14:49.525641 20430 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 
W20260709 12:14:49.526181 20428 leader_election.cc:336] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:49.526419 20673 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:49.526892 20673 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.527267 20673 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:49.534880 20673 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:49.536862 20673 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:49.537909 20538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:49.538698 20538 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:49.540499 20428 leader_election.cc:336] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111)
I20260709 12:14:49.545400 20538 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:49.546818 20430 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, d022a54d46334c568c045b076b6ceed8; no voters: 9142a6f9cf1742d6a7e2e5f53ef197a4
I20260709 12:14:49.547665 20673 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:49.548226 20673 raft_consensus.cc:697] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:49.549130 20673 consensus_queue.cc:237] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:49.563536 20286 catalog_manager.cc:5697] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 1 to 3, leader changed from 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65) to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 3 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:49.679927 20428 consensus_peers.cc:597] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:50.035557 20538 raft_consensus.cc:1275] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:14:50.037137 20673 consensus_queue.cc:1048] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:14:50.052054 20428 consensus_peers.cc:597] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:50.198346 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:14:51.156342 20428 consensus_peers.cc:597] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:51.199615 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:14:51.358878 20428 consensus_peers.cc:597] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.361681 20502 consensus_peers.cc:597] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.491289 20502 consensus_peers.cc:597] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.632791 20428 consensus_peers.cc:597] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.684613 20428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111) [suppressed 103 similar messages]
W20260709 12:14:51.687093 20428 consensus_peers.cc:597] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.731130 20428 consensus_peers.cc:597] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:51.735000  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:51.743750 20695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:51.744129 20696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:51.746110 20698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:51.748201  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:51.749318  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:51.749531  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:51.749676  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599291749665 us; error 0 us; skew 500 ppm
I20260709 12:14:51.750180  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:51.752991  9261 webserver.cc:533] Webserver started at http://127.9.11.65:35257/ using document root <none> and password file <none>
I20260709 12:14:51.753476  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:51.753638  9261 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:14:51.757653  9261 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:14:51.762212 20703 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:51.763221  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 12:14:51.763542  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
format_stamp: "Formatted at 2026-07-09 12:14:44 on dist-test-slave-k289"
I20260709 12:14:51.763969  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:51.777974  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:51.779371  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:51.781174  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:51.785889 20710 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:14:51.845382  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:14:51.845750  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.061s	user 0.003s	sys 0.000s
I20260709 12:14:51.846120  9261 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:14:51.851428 20710 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:51.869287 20710 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:51.870154 20710 tablet_bootstrap.cc:492] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:51.870759 20710 ts_tablet_manager.cc:1403] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.000s
I20260709 12:14:51.873428 20710 raft_consensus.cc:359] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.874064 20710 raft_consensus.cc:740] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:51.874842 20710 consensus_queue.cc:260] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.877315 20710 ts_tablet_manager.cc:1434] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:14:51.878077 20710 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:51.892216 20710 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:51.893080 20710 tablet_bootstrap.cc:492] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:51.893673 20710 ts_tablet_manager.cc:1403] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.008s
I20260709 12:14:51.894377  9261 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:14:51.894733  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.049s	user 0.042s	sys 0.004s
I20260709 12:14:51.896348 20710 raft_consensus.cc:359] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.897011 20710 raft_consensus.cc:740] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:51.897706 20710 consensus_queue.cc:260] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.899471 20710 ts_tablet_manager.cc:1434] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:14:51.900215 20710 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:51.917023 20710 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:51.918408 20710 tablet_bootstrap.cc:492] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:51.919345 20710 ts_tablet_manager.cc:1403] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:14:51.922410 20710 raft_consensus.cc:359] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.923152 20710 raft_consensus.cc:740] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:51.923951 20710 consensus_queue.cc:260] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.925758 20710 ts_tablet_manager.cc:1434] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:14:51.926518 20710 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:51.939817 20710 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:51.940697 20710 tablet_bootstrap.cc:492] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:51.941496 20710 ts_tablet_manager.cc:1403] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:14:51.943781 20710 raft_consensus.cc:359] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.944344 20710 raft_consensus.cc:740] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:51.945037 20710 consensus_queue.cc:260] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.947028 20710 ts_tablet_manager.cc:1434] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:14:51.947758 20710 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:51.962926 20710 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:51.964308 20710 tablet_bootstrap.cc:492] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:51.965276 20710 ts_tablet_manager.cc:1403] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:14:51.968336  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:42053
I20260709 12:14:51.968504 20772 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:42053 every 8 connection(s)
I20260709 12:14:51.969019 20710 raft_consensus.cc:359] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.969854 20710 raft_consensus.cc:740] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:51.970670 20710 consensus_queue.cc:260] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:51.973515 20710 ts_tablet_manager.cc:1434] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:14:51.974762 20710 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:51.975279  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:51.998868 20773 heartbeater.cc:344] Connected to a master server at 127.9.11.126:38165
I20260709 12:14:51.999408 20773 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:51.999859 20710 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:52.000383 20773 heartbeater.cc:507] Master 127.9.11.126:38165 requested a full tablet report, sending...
I20260709 12:14:52.001546 20710 tablet_bootstrap.cc:492] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:52.002322 20710 ts_tablet_manager.cc:1403] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.028s	user 0.025s	sys 0.000s
I20260709 12:14:52.004562 20463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:52.005467 20463 raft_consensus.cc:606] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:52.005517 20710 raft_consensus.cc:359] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:52.006352 20710 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:52.007292 20710 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:52.008762 20463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c13381f98574253ba7377725c8391ee"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:52.009331 20710 ts_tablet_manager.cc:1434] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:14:52.009459 20463 raft_consensus.cc:606] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:52.010201 20710 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:52.010516 20286 ts_manager.cc:194] Re-registered known tserver with Master: 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053)
I20260709 12:14:52.012674 20463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f8cff5054984073936d856d110a8a94"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:52.013347 20463 raft_consensus.cc:606] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:52.016297 20463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:52.016886 20463 raft_consensus.cc:606] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:52.018079  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:14:52.019037  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:52.019888  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:52.022426 20286 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36966
I20260709 12:14:52.025465 20710 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:52.026239 20710 tablet_bootstrap.cc:492] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:52.026859 20710 ts_tablet_manager.cc:1403] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:14:52.027032 20773 heartbeater.cc:499] Master 127.9.11.126:38165 was elected leader, sending a full tablet report...
I20260709 12:14:52.029825 20710 raft_consensus.cc:359] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.030669 20710 raft_consensus.cc:740] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:52.031880 20710 consensus_queue.cc:260] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.034284 20710 ts_tablet_manager.cc:1434] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:14:52.035048 20710 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:52.047766 20747 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:14:52.049732 20428 consensus_peers.cc:597] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:52.051613 20710 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:52.052754 20710 tablet_bootstrap.cc:492] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:52.053596 20710 ts_tablet_manager.cc:1403] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:14:52.056339 20710 raft_consensus.cc:359] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.057137 20710 raft_consensus.cc:740] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:52.057919 20710 consensus_queue.cc:260] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.059908 20710 ts_tablet_manager.cc:1434] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:14:52.060731 20710 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:52.074503 20747 raft_consensus.cc:3060] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.077338 20710 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:52.078262 20710 tablet_bootstrap.cc:492] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:52.078871 20710 ts_tablet_manager.cc:1403] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:14:52.081218 20710 raft_consensus.cc:359] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.081784 20710 raft_consensus.cc:740] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:52.082758 20710 consensus_queue.cc:260] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.085172 20710 ts_tablet_manager.cc:1434] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:14:52.086207 20710 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap starting.
I20260709 12:14:52.104290 20710 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:14:52.105510 20710 tablet_bootstrap.cc:492] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Bootstrap complete.
I20260709 12:14:52.106310 20710 ts_tablet_manager.cc:1403] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 12:14:52.108763 20710 raft_consensus.cc:359] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.109323 20710 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9142a6f9cf1742d6a7e2e5f53ef197a4, State: Initialized, Role: FOLLOWER
I20260709 12:14:52.109949 20710 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.111943 20710 ts_tablet_manager.cc:1434] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
W20260709 12:14:52.200819 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:52.286396 20673 raft_consensus.cc:993] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.287192 20788 raft_consensus.cc:1081] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.289155 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:52.289914 20539 raft_consensus.cc:493] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:52.290410 20539 raft_consensus.cc:3060] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.297946 20539 raft_consensus.cc:515] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.300187 20539 leader_election.cc:290] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:52.301235 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:52.301405 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c0c3180a764d39bf327bd26b40afb2" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:52.302287 20747 raft_consensus.cc:3060] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.302335 20463 raft_consensus.cc:3055] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:52.302860 20463 raft_consensus.cc:740] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:52.303678 20463 consensus_queue.cc:260] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.304826 20463 raft_consensus.cc:3060] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.309229 20747 raft_consensus.cc:2468] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:52.310628 20502 leader_election.cc:304] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:52.311591 20613 raft_consensus.cc:2804] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:52.311735 20463 raft_consensus.cc:2468] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:52.313959 20613 raft_consensus.cc:697] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:52.315038 20613 consensus_queue.cc:237] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.321100 20788 raft_consensus.cc:993] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.321601 20673 raft_consensus.cc:1081] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.325390 20287 catalog_manager.cc:5697] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 1 to 2, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.326969 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:52.327847 20539 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:52.328214 20539 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.334702 20539 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.337261 20539 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:52.337973 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:52.338027 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:52.339049 20747 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.339061 20463 raft_consensus.cc:3055] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:52.339725 20463 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:52.340826 20463 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.343189 20463 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.346123 20788 raft_consensus.cc:993] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.346781 20673 raft_consensus.cc:1081] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.348309 20747 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:52.348701 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f8cff5054984073936d856d110a8a94"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:52.349361 20539 raft_consensus.cc:493] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:52.349610 20502 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:52.349748 20539 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:52.350639 20613 raft_consensus.cc:2804] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:52.351217 20613 raft_consensus.cc:697] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:52.351303 20463 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:52.352324 20613 consensus_queue.cc:237] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.357378 20539 raft_consensus.cc:515] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.359891 20539 leader_election.cc:290] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 4 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:52.361172 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:52.361545 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8cff5054984073936d856d110a8a94" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:52.361985 20463 raft_consensus.cc:3055] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:52.361631 20287 catalog_manager.cc:5697] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 1 to 2, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.362278 20747 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:52.362417 20463 raft_consensus.cc:740] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:52.363651 20463 consensus_queue.cc:260] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.365240 20463 raft_consensus.cc:3060] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:52.371630 20747 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 4.
I20260709 12:14:52.373076 20502 leader_election.cc:304] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:52.374150 20613 raft_consensus.cc:2804] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:52.374686 20613 raft_consensus.cc:697] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:52.375111 20463 raft_consensus.cc:2468] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 4.
I20260709 12:14:52.375643 20613 consensus_queue.cc:237] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.384472 20287 catalog_manager.cc:5697] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 3 to 4, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 4 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.454479 20788 raft_consensus.cc:993] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.455070 20788 raft_consensus.cc:1081] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:52.456655 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c13381f98574253ba7377725c8391ee"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:52.457217 20539 raft_consensus.cc:493] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:52.457512 20539 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.463753 20539 raft_consensus.cc:515] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.465848 20539 leader_election.cc:290] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:52.466969 20463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:52.467186 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:52.467752 20463 raft_consensus.cc:3055] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:52.467832 20747 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.468104 20463 raft_consensus.cc:740] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:52.469097 20463 consensus_queue.cc:260] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.470518 20463 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:52.474052 20747 raft_consensus.cc:2468] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:52.475322 20502 leader_election.cc:304] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:52.476150 20613 raft_consensus.cc:2804] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:52.476568 20613 raft_consensus.cc:697] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:52.477305 20463 raft_consensus.cc:2468] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:52.477392 20613 consensus_queue.cc:237] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:52.485495 20287 catalog_manager.cc:5697] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 1 to 2, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:52.722333 20747 raft_consensus.cc:1275] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:52.723791 20613 consensus_queue.cc:1048] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:52.744483 20463 raft_consensus.cc:1275] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:52.746250 20799 consensus_queue.cc:1048] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:52.852183 20747 raft_consensus.cc:1275] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:52.853863 20799 consensus_queue.cc:1048] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:52.862756 20463 raft_consensus.cc:1275] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:52.867062 20789 consensus_queue.cc:1048] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:52.892092 20747 raft_consensus.cc:1275] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:52.893543 20789 consensus_queue.cc:1048] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:52.900815 20463 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:52.903184 20613 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:52.910233 20463 raft_consensus.cc:1275] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:52.913957 20613 consensus_queue.cc:1048] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:52.920029 20747 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:52.922509 20809 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:53.032974  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:53.052608 20539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c13381f98574253ba7377725c8391ee"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
mode: GRACEFUL
new_leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:59600
I20260709 12:14:53.053269 20539 raft_consensus.cc:606] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Received request to transfer leadership to TS 4987bb792cfe49b8bdc19fe144b5ec1a
I20260709 12:14:53.056272 20539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
mode: GRACEFUL
new_leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:59600
I20260709 12:14:53.056883 20539 raft_consensus.cc:606] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Received request to transfer leadership to TS 4987bb792cfe49b8bdc19fe144b5ec1a
I20260709 12:14:53.058048  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:14:53.058670  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:53.059466  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:53.201949 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:53.354717 20808 raft_consensus.cc:993] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: : Instructing follower 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:53.355607 20809 raft_consensus.cc:1081] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Signalling peer 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:53.357398 20465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c13381f98574253ba7377725c8391ee"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:36818
I20260709 12:14:53.357988 20465 raft_consensus.cc:493] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:53.358377 20465 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:53.365123 20465 raft_consensus.cc:515] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:53.367408 20465 leader_election.cc:290] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:53.368525 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:53.368639 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c13381f98574253ba7377725c8391ee" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:53.369350 20747 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:53.369405 20539 raft_consensus.cc:3055] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:53.369848 20539 raft_consensus.cc:740] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:53.370628 20539 consensus_queue.cc:260] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:53.371857 20539 raft_consensus.cc:3060] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:53.376475 20747 raft_consensus.cc:2468] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:53.378111 20428 leader_election.cc:304] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:53.378778 20539 raft_consensus.cc:2468] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:53.379189 20803 raft_consensus.cc:2804] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:53.380569 20803 raft_consensus.cc:697] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:53.381560 20803 consensus_queue.cc:237] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:53.390156 20285 catalog_manager.cc:5697] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 2 to 3, leader changed from d022a54d46334c568c045b076b6ceed8 (127.9.11.67) to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 3 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:53.544477 20808 raft_consensus.cc:993] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: : Instructing follower 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:53.545042 20613 raft_consensus.cc:1081] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Signalling peer 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:53.546667 20465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:36818
I20260709 12:14:53.547236 20465 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:53.547510 20465 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:53.556015 20465 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:53.557938 20465 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:53.559046 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:53.559046 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:53.560045 20539 raft_consensus.cc:3055] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:53.560079 20747 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:53.560508 20539 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:53.561491 20539 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:53.562940 20539 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:53.568787 20747 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:53.570484 20428 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:53.571376 20803 raft_consensus.cc:2804] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:53.571316 20539 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:53.572129 20803 raft_consensus.cc:697] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:53.573112 20803 consensus_queue.cc:237] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:53.580355 20285 catalog_manager.cc:5697] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 2 to 3, leader changed from d022a54d46334c568c045b076b6ceed8 (127.9.11.67) to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 3 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:53.800621 20747 raft_consensus.cc:1275] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:53.802114 20803 consensus_queue.cc:1048] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:53.812156 20539 raft_consensus.cc:1275] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:53.815307 20803 consensus_queue.cc:1048] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:54.066304  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:54.074488 20465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:54.075165 20465 raft_consensus.cc:606] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:54.077920 20465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:54.078449 20465 raft_consensus.cc:606] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:54.079911  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:14:54.080621  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:54.081059 20747 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:54.081358  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:54.082580 20803 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:54.093598 20539 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:54.094977 20815 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:54.107841 20824 raft_consensus.cc:993] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:54.108462 20815 raft_consensus.cc:1081] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:54.109999 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:54.110575 20539 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:54.110901 20539 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:54.117063 20539 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:54.118649 20539 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 4 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:54.119828 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:54.119954 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:54.120555 20465 raft_consensus.cc:3055] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:54.120615 20747 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:54.120879 20465 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:54.121703 20465 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:54.122833 20465 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:54.127036 20747 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 4.
I20260709 12:14:54.128319 20502 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:54.129169 20465 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 4.
I20260709 12:14:54.129235 20613 raft_consensus.cc:2804] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:54.130823 20613 raft_consensus.cc:697] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:54.131604 20613 consensus_queue.cc:237] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:54.138036 20286 catalog_manager.cc:5697] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 3 to 4, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 4 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:54.203402 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:54.566713 20465 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:54.568349 20613 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:54.576858 20747 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:54.578817 20613 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:54.637271 20803 raft_consensus.cc:993] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:54.637894 20815 raft_consensus.cc:1081] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:54.639541 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:54.640077 20539 raft_consensus.cc:493] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:54.640393 20539 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:54.646631 20539 raft_consensus.cc:515] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:54.648468 20539 leader_election.cc:290] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:54.649856 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:54.650030 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:54.650632 20465 raft_consensus.cc:3055] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:54.650693 20747 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:54.651014 20465 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:54.651942 20465 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:54.653378 20465 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:54.657825 20747 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:54.659178 20502 leader_election.cc:304] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:54.660225 20613 raft_consensus.cc:2804] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:54.660557 20465 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:54.660718 20613 raft_consensus.cc:697] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:54.662212 20613 consensus_queue.cc:237] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:54.669975 20285 catalog_manager.cc:5697] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 1 to 2, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:55.091185  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:55.099457 20539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
mode: GRACEFUL
new_leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:59600
I20260709 12:14:55.100080 20539 raft_consensus.cc:606] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Received request to transfer leadership to TS 4987bb792cfe49b8bdc19fe144b5ec1a
I20260709 12:14:55.102667 20539 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
mode: GRACEFUL
new_leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:59600
I20260709 12:14:55.103185 20539 raft_consensus.cc:606] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Received request to transfer leadership to TS 4987bb792cfe49b8bdc19fe144b5ec1a
I20260709 12:14:55.104206  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:14:55.104805  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:55.105444  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:55.125257 20746 raft_consensus.cc:1275] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:55.127125 20613 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:55.128638 20465 raft_consensus.cc:1275] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:55.130455 20613 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:55.150614 20835 raft_consensus.cc:993] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: : Instructing follower 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:55.151317 20613 raft_consensus.cc:1081] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Signalling peer 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:55.153759 20465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:36818
I20260709 12:14:55.154470 20465 raft_consensus.cc:493] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:55.155506 20465 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:55.164054 20465 raft_consensus.cc:515] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:55.166783 20465 leader_election.cc:290] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:55.166934 20746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:55.167850 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:55.167909 20746 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:55.168530 20539 raft_consensus.cc:3055] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:55.168817 20539 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:55.169468 20539 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:55.170511 20539 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:55.174664 20746 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:55.176017 20428 leader_election.cc:304] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:55.176625 20539 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 3.
I20260709 12:14:55.176934 20815 raft_consensus.cc:2804] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:55.178164 20815 raft_consensus.cc:697] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:55.179180 20815 consensus_queue.cc:237] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:55.187175 20286 catalog_manager.cc:5697] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 2 to 3, leader changed from d022a54d46334c568c045b076b6ceed8 (127.9.11.67) to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 3 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:55.231519 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:55.589339 20613 raft_consensus.cc:993] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: : Instructing follower 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:55.590211 20835 raft_consensus.cc:1081] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Signalling peer 4987bb792cfe49b8bdc19fe144b5ec1a to start an election
I20260709 12:14:55.592244 20465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
 from {username='slave'} at 127.0.0.1:36818
I20260709 12:14:55.592912 20465 raft_consensus.cc:493] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:55.593290 20465 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:55.604555 20465 raft_consensus.cc:515] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:55.606638 20465 leader_election.cc:290] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 5 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), d022a54d46334c568c045b076b6ceed8 (127.9.11.67:33443)
I20260709 12:14:55.608079 20539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d022a54d46334c568c045b076b6ceed8"
I20260709 12:14:55.607889 20746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dff3959e3c6246cd96ee165ee236a06d" candidate_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:55.608704 20539 raft_consensus.cc:3055] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:55.608806 20746 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:55.609009 20539 raft_consensus.cc:740] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:55.610199 20539 consensus_queue.cc:260] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:55.611474 20539 raft_consensus.cc:3060] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:55.618029 20746 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 5.
I20260709 12:14:55.619606 20539 raft_consensus.cc:2468] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4987bb792cfe49b8bdc19fe144b5ec1a in term 5.
I20260709 12:14:55.619745 20428 leader_election.cc:304] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4987bb792cfe49b8bdc19fe144b5ec1a, 9142a6f9cf1742d6a7e2e5f53ef197a4; no voters: 
I20260709 12:14:55.621074 20815 raft_consensus.cc:2804] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:55.621547 20815 raft_consensus.cc:697] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 5 LEADER]: Becoming Leader. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:55.622857 20815 consensus_queue.cc:237] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:55.632897 20286 catalog_manager.cc:5697] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a reported cstate change: term changed from 4 to 5, leader changed from d022a54d46334c568c045b076b6ceed8 (127.9.11.67) to 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66). New cstate: current_term: 5 leader_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:55.647645 20539 raft_consensus.cc:1275] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:55.649663 20815 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:55.661695 20747 raft_consensus.cc:1275] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:55.664613 20842 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:56.048688 20538 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 5 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:56.050294 20850 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:56.063748 20746 raft_consensus.cc:1275] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 5 FOLLOWER]: Refusing update from remote peer 4987bb792cfe49b8bdc19fe144b5ec1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:56.066044 20815 consensus_queue.cc:1048] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:56.112982  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:56.124663 20465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:56.125507 20465 raft_consensus.cc:606] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:56.128571 20465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11"
dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
mode: GRACEFUL
new_leader_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:35782
I20260709 12:14:56.129133 20465 raft_consensus.cc:606] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Received request to transfer leadership to TS d022a54d46334c568c045b076b6ceed8
I20260709 12:14:56.130328  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:14:56.131006  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:56.131850  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:56.220927 20850 raft_consensus.cc:993] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:56.221686 20815 raft_consensus.cc:1081] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:56.223505 20538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:56.224107 20538 raft_consensus.cc:493] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:56.224429 20538 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:56.233824 20538 raft_consensus.cc:515] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
W20260709 12:14:56.234836 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:56.236155 20538 leader_election.cc:290] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 4 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:56.237124 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:56.237385 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f18649d815484e82b8c54d7718cecc11" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:56.237957 20747 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:56.238081 20465 raft_consensus.cc:3055] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:56.238513 20465 raft_consensus.cc:740] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:56.239502 20465 consensus_queue.cc:260] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:56.240667 20465 raft_consensus.cc:3060] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:56.246374 20747 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 4.
I20260709 12:14:56.247807 20502 leader_election.cc:304] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:56.248553 20465 raft_consensus.cc:2468] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 4.
I20260709 12:14:56.248682 20835 raft_consensus.cc:2804] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:56.249287 20835 raft_consensus.cc:697] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:56.250367 20835 consensus_queue.cc:237] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } }
I20260709 12:14:56.258209 20285 catalog_manager.cc:5697] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 3 to 4, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 4 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:56.311945 20842 raft_consensus.cc:993] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: : Instructing follower d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:56.312605 20815 raft_consensus.cc:1081] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Signalling peer d022a54d46334c568c045b076b6ceed8 to start an election
I20260709 12:14:56.314535 20539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4"
dest_uuid: "d022a54d46334c568c045b076b6ceed8"
 from {username='slave'} at 127.0.0.1:38400
I20260709 12:14:56.315176 20539 raft_consensus.cc:493] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:56.315523 20539 raft_consensus.cc:3060] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:56.322650 20539 raft_consensus.cc:515] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:56.324528 20539 leader_election.cc:290] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Requested vote from peers 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053), 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035)
I20260709 12:14:56.325477 20747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4"
I20260709 12:14:56.325698 20465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a927a0941f34205b9c4c622b45168e4" candidate_uuid: "d022a54d46334c568c045b076b6ceed8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a"
I20260709 12:14:56.326290 20747 raft_consensus.cc:3060] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:56.326352 20465 raft_consensus.cc:3055] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:56.326756 20465 raft_consensus.cc:740] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4987bb792cfe49b8bdc19fe144b5ec1a, State: Running, Role: LEADER
I20260709 12:14:56.327975 20465 consensus_queue.cc:260] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:56.329553 20465 raft_consensus.cc:3060] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:56.336145 20747 raft_consensus.cc:2468] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:56.337556 20502 leader_election.cc:304] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9142a6f9cf1742d6a7e2e5f53ef197a4, d022a54d46334c568c045b076b6ceed8; no voters: 
I20260709 12:14:56.338647 20835 raft_consensus.cc:2804] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:56.338809 20465 raft_consensus.cc:2468] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d022a54d46334c568c045b076b6ceed8 in term 2.
I20260709 12:14:56.339207 20835 raft_consensus.cc:697] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Becoming Leader. State: Replica: d022a54d46334c568c045b076b6ceed8, State: Running, Role: LEADER
I20260709 12:14:56.340442 20835 consensus_queue.cc:237] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } }
I20260709 12:14:56.347946 20285 catalog_manager.cc:5697] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 reported cstate change: term changed from 1 to 2, leader changed from 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66) to d022a54d46334c568c045b076b6ceed8 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "d022a54d46334c568c045b076b6ceed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d022a54d46334c568c045b076b6ceed8" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33443 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:56.682641 20465 raft_consensus.cc:1275] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:56.684001 20613 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:56.700937 20747 raft_consensus.cc:1275] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:56.705526 20835 consensus_queue.cc:1048] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:56.794034 20465 raft_consensus.cc:1275] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:56.795672 20835 consensus_queue.cc:1048] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4987bb792cfe49b8bdc19fe144b5ec1a" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:14:56.810211 20747 raft_consensus.cc:1275] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Refusing update from remote peer d022a54d46334c568c045b076b6ceed8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:56.812474 20613 consensus_queue.cc:1048] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9142a6f9cf1742d6a7e2e5f53ef197a4" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:57.138332  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:57.173437  9261 tablet_server.cc:179] TabletServer@127.9.11.65:42053 shutting down...
W20260709 12:14:57.186573 20428 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 12:14:57.196039 20428 consensus_peers.cc:597] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.198247 20502 consensus_peers.cc:597] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.210098  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:57.211336  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.212419  9261 raft_consensus.cc:2243] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.213027  9261 raft_consensus.cc:2272] T 016522e4a22b4ef4969964155f73bae3 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.216696  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.217833  9261 raft_consensus.cc:2243] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.218719  9261 raft_consensus.cc:2272] T b28754f70ce84703a760f5614f45ccf8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.223323  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.224416  9261 raft_consensus.cc:2243] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.225081  9261 raft_consensus.cc:2272] T f18649d815484e82b8c54d7718cecc11 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.229238  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.230010  9261 raft_consensus.cc:2243] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.230521  9261 raft_consensus.cc:2272] T 02929838c5ed4c25b31cc52587d022da P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.233389  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.234045  9261 raft_consensus.cc:2243] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.234591  9261 raft_consensus.cc:2272] T dff3959e3c6246cd96ee165ee236a06d P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.236146 20336 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:14:57.237434  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.238274  9261 raft_consensus.cc:2243] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.238782  9261 raft_consensus.cc:2272] T 5a927a0941f34205b9c4c622b45168e4 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.243305  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.244112  9261 raft_consensus.cc:2243] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.244685  9261 raft_consensus.cc:2272] T 9f8cff5054984073936d856d110a8a94 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.247496  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.248071  9261 raft_consensus.cc:2243] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.248624  9261 raft_consensus.cc:2272] T 6c13381f98574253ba7377725c8391ee P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.251437  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.252038  9261 raft_consensus.cc:2243] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.252650  9261 raft_consensus.cc:2272] T 043e3ceed12e4dbba399d7e6930749b8 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.255609  9261 tablet_replica.cc:333] stopping tablet replica
I20260709 12:14:57.256220  9261 raft_consensus.cc:2243] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.256927  9261 raft_consensus.cc:2272] T 14c0c3180a764d39bf327bd26b40afb2 P 9142a6f9cf1742d6a7e2e5f53ef197a4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.261176 20502 consensus_peers.cc:597] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.296615  9261 tablet_server.cc:196] TabletServer@127.9.11.65:42053 shutdown complete.
W20260709 12:14:57.308485 20428 consensus_peers.cc:597] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.332360 20502 consensus_peers.cc:597] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.339764  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
W20260709 12:14:57.354957 20502 consensus_peers.cc:597] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 -> Peer 9142a6f9cf1742d6a7e2e5f53ef197a4 (127.9.11.65:42053): Couldn't send request to peer 9142a6f9cf1742d6a7e2e5f53ef197a4. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:42053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.367777 20504 consensus_peers.cc:597] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 -> Peer 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035): Couldn't send request to peer 4987bb792cfe49b8bdc19fe144b5ec1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.375465  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:57.376609  9261 tablet_replica.cc:333] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.378065  9261 raft_consensus.cc:2243] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.379578  9261 raft_consensus.cc:2272] T 02929838c5ed4c25b31cc52587d022da P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.382906  9261 tablet_replica.cc:333] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.383741  9261 raft_consensus.cc:2243] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.384241  9261 raft_consensus.cc:2272] T 5a927a0941f34205b9c4c622b45168e4 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.387148  9261 tablet_replica.cc:333] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.387876  9261 raft_consensus.cc:2243] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.388813  9261 raft_consensus.cc:2272] T dff3959e3c6246cd96ee165ee236a06d P 4987bb792cfe49b8bdc19fe144b5ec1a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.391523  9261 tablet_replica.cc:333] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.392230  9261 raft_consensus.cc:2243] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.392737  9261 raft_consensus.cc:2272] T 016522e4a22b4ef4969964155f73bae3 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.395463  9261 tablet_replica.cc:333] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.396057  9261 raft_consensus.cc:2243] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.397161  9261 raft_consensus.cc:2272] T 043e3ceed12e4dbba399d7e6930749b8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.399958  9261 tablet_replica.cc:333] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.400696  9261 raft_consensus.cc:2243] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.401242  9261 raft_consensus.cc:2272] T 14c0c3180a764d39bf327bd26b40afb2 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.403832  9261 tablet_replica.cc:333] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.404506  9261 raft_consensus.cc:2243] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.405073  9261 raft_consensus.cc:2272] T b28754f70ce84703a760f5614f45ccf8 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.407693  9261 tablet_replica.cc:333] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.408325  9261 raft_consensus.cc:2243] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.408795  9261 raft_consensus.cc:2272] T f18649d815484e82b8c54d7718cecc11 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.412027  9261 tablet_replica.cc:333] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.412619  9261 raft_consensus.cc:2243] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.413713  9261 raft_consensus.cc:2272] T 6c13381f98574253ba7377725c8391ee P 4987bb792cfe49b8bdc19fe144b5ec1a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.416157  9261 tablet_replica.cc:333] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a: stopping tablet replica
I20260709 12:14:57.416714  9261 raft_consensus.cc:2243] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.417253  9261 raft_consensus.cc:2272] T 9f8cff5054984073936d856d110a8a94 P 4987bb792cfe49b8bdc19fe144b5ec1a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.454715  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
I20260709 12:14:57.489706  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
W20260709 12:14:57.505903 20504 consensus_peers.cc:597] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 -> Peer 4987bb792cfe49b8bdc19fe144b5ec1a (127.9.11.66:35035): Couldn't send request to peer 4987bb792cfe49b8bdc19fe144b5ec1a. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:35035: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.522451  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:57.523590  9261 tablet_replica.cc:333] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.524454  9261 raft_consensus.cc:2243] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.525056  9261 raft_consensus.cc:2272] T 02929838c5ed4c25b31cc52587d022da P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.527992  9261 tablet_replica.cc:333] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.528692  9261 raft_consensus.cc:2243] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.529834  9261 raft_consensus.cc:2272] T 9f8cff5054984073936d856d110a8a94 P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.532815  9261 tablet_replica.cc:333] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.533515  9261 raft_consensus.cc:2243] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.534065  9261 raft_consensus.cc:2272] T 6c13381f98574253ba7377725c8391ee P d022a54d46334c568c045b076b6ceed8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.536669  9261 tablet_replica.cc:333] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.537312  9261 raft_consensus.cc:2243] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.538350  9261 raft_consensus.cc:2272] T 14c0c3180a764d39bf327bd26b40afb2 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.540961  9261 tablet_replica.cc:333] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.541623  9261 raft_consensus.cc:2243] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.542177  9261 raft_consensus.cc:2272] T 043e3ceed12e4dbba399d7e6930749b8 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.545280  9261 tablet_replica.cc:333] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.545949  9261 raft_consensus.cc:2243] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.547029  9261 raft_consensus.cc:2272] T 016522e4a22b4ef4969964155f73bae3 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.549634  9261 tablet_replica.cc:333] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.550252  9261 raft_consensus.cc:2243] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.551369  9261 raft_consensus.cc:2272] T b28754f70ce84703a760f5614f45ccf8 P d022a54d46334c568c045b076b6ceed8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.554023  9261 tablet_replica.cc:333] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.554665  9261 raft_consensus.cc:2243] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.555694  9261 raft_consensus.cc:2272] T f18649d815484e82b8c54d7718cecc11 P d022a54d46334c568c045b076b6ceed8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.559508  9261 tablet_replica.cc:333] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.560113  9261 raft_consensus.cc:2243] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.561066  9261 raft_consensus.cc:2272] T 5a927a0941f34205b9c4c622b45168e4 P d022a54d46334c568c045b076b6ceed8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.564344  9261 tablet_replica.cc:333] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8: stopping tablet replica
I20260709 12:14:57.564973  9261 raft_consensus.cc:2243] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.565683  9261 raft_consensus.cc:2272] T dff3959e3c6246cd96ee165ee236a06d P d022a54d46334c568c045b076b6ceed8 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.607856  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:14:57.644232  9261 master.cc:562] Master@127.9.11.126:38165 shutting down...
I20260709 12:14:57.674374  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.675197  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.675597  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P 47f3305b26394df89f9f32e82514e915: stopping tablet replica
I20260709 12:14:57.699223  9261 master.cc:584] Master@127.9.11.126:38165 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13790 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:14:57.754675  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:34829
I20260709 12:14:57.756122  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:57.762713 20876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:57.763104 20877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:57.764146 20879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:57.764621  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:57.765872  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:57.766060  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:57.766178  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599297766166 us; error 0 us; skew 500 ppm
I20260709 12:14:57.766690  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:57.769901  9261 webserver.cc:533] Webserver started at http://127.9.11.126:45203/ using document root <none> and password file <none>
I20260709 12:14:57.770411  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:57.770579  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:57.770823  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:57.772145  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "c4fa74e589564011aacc08dd74e7dbd7"
format_stamp: "Formatted at 2026-07-09 12:14:57 on dist-test-slave-k289"
I20260709 12:14:57.776890  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:14:57.780580 20884 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:57.781399  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:14:57.781706  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "c4fa74e589564011aacc08dd74e7dbd7"
format_stamp: "Formatted at 2026-07-09 12:14:57 on dist-test-slave-k289"
I20260709 12:14:57.782027  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:57.813012  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:57.814733  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:57.865448  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:34829
I20260709 12:14:57.865585 20935 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:34829 every 8 connection(s)
I20260709 12:14:57.870210 20936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:57.885360 20936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7: Bootstrap starting.
I20260709 12:14:57.891650 20936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:57.896763 20936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7: No bootstrap required, opened a new log
I20260709 12:14:57.899479 20936 raft_consensus.cc:359] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4fa74e589564011aacc08dd74e7dbd7" member_type: VOTER }
I20260709 12:14:57.899909 20936 raft_consensus.cc:385] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:57.900111 20936 raft_consensus.cc:740] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fa74e589564011aacc08dd74e7dbd7, State: Initialized, Role: FOLLOWER
I20260709 12:14:57.900774 20936 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4fa74e589564011aacc08dd74e7dbd7" member_type: VOTER }
I20260709 12:14:57.901288 20936 raft_consensus.cc:399] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:14:57.901505 20936 raft_consensus.cc:493] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:14:57.901728 20936 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:57.907924 20936 raft_consensus.cc:515] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4fa74e589564011aacc08dd74e7dbd7" member_type: VOTER }
I20260709 12:14:57.908478 20936 leader_election.cc:304] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4fa74e589564011aacc08dd74e7dbd7; no voters: 
I20260709 12:14:57.909757 20936 leader_election.cc:290] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:14:57.910234 20939 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:57.911827 20939 raft_consensus.cc:697] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 1 LEADER]: Becoming Leader. State: Replica: c4fa74e589564011aacc08dd74e7dbd7, State: Running, Role: LEADER
I20260709 12:14:57.912607 20939 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4fa74e589564011aacc08dd74e7dbd7" member_type: VOTER }
I20260709 12:14:57.913215 20936 sys_catalog.cc:565] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:14:57.917629 20940 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4fa74e589564011aacc08dd74e7dbd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4fa74e589564011aacc08dd74e7dbd7" member_type: VOTER } }
I20260709 12:14:57.917804 20941 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4fa74e589564011aacc08dd74e7dbd7. Latest consensus state: current_term: 1 leader_uuid: "c4fa74e589564011aacc08dd74e7dbd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4fa74e589564011aacc08dd74e7dbd7" member_type: VOTER } }
I20260709 12:14:57.918673 20940 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [sys.catalog]: This master's current role is: LEADER
I20260709 12:14:57.918699 20941 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [sys.catalog]: This master's current role is: LEADER
I20260709 12:14:57.921211 20945 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:14:57.926741 20945 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:14:57.932399  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:14:57.937777 20945 catalog_manager.cc:1383] Generated new cluster ID: 8325e250f9eb49eb92488ff34f411620
I20260709 12:14:57.938220 20945 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:14:57.950747 20945 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:14:57.952284 20945 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:14:57.969521 20945 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7: Generated new TSK 0
I20260709 12:14:57.970268 20945 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:14:58.004673  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:58.012217 20957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:58.013428 20958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:58.015185 20960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:58.015913  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:58.017069  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:58.017262  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:58.017504  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599298017480 us; error 0 us; skew 500 ppm
I20260709 12:14:58.018146  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:58.020895  9261 webserver.cc:533] Webserver started at http://127.9.11.65:33833/ using document root <none> and password file <none>
I20260709 12:14:58.021508  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:58.021723  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:58.022046  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:58.023452  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "5739a93626c8432aa7a9c5381950f843"
format_stamp: "Formatted at 2026-07-09 12:14:58 on dist-test-slave-k289"
I20260709 12:14:58.028667  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:14:58.032454 20965 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.033349  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:14:58.033660  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "5739a93626c8432aa7a9c5381950f843"
format_stamp: "Formatted at 2026-07-09 12:14:58 on dist-test-slave-k289"
I20260709 12:14:58.033986  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:58.050446  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:58.051990  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:58.053679  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:58.056337  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:58.056579  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.056844  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:58.057049  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.106761  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:35217
I20260709 12:14:58.106914 21027 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:35217 every 8 connection(s)
I20260709 12:14:58.113584  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:58.122658 21032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:58.123409 21033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:58.128880 21035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:58.130381 21028 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34829
I20260709 12:14:58.130681  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:58.130944 21028 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:58.132010 21028 heartbeater.cc:507] Master 127.9.11.126:34829 requested a full tablet report, sending...
I20260709 12:14:58.132193  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:58.132526  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:58.132721  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599298132703 us; error 0 us; skew 500 ppm
I20260709 12:14:58.133617  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:58.134721 20901 ts_manager.cc:194] Registered new tserver with Master: 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217)
I20260709 12:14:58.136927  9261 webserver.cc:533] Webserver started at http://127.9.11.66:44173/ using document root <none> and password file <none>
I20260709 12:14:58.137542  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:58.137785  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:58.137632 20901 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39984
I20260709 12:14:58.138334  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:58.140708  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "1e102e6a42244f128030652534b1d52d"
format_stamp: "Formatted at 2026-07-09 12:14:58 on dist-test-slave-k289"
I20260709 12:14:58.150399  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:14:58.157112 21040 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.158360  9261 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:14:58.158735  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "1e102e6a42244f128030652534b1d52d"
format_stamp: "Formatted at 2026-07-09 12:14:58 on dist-test-slave-k289"
I20260709 12:14:58.159205  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:58.199966  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:58.201483  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:58.203280  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:58.205875  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:58.206138  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.206400  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:58.206586  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.258278  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:34331
I20260709 12:14:58.258442 21102 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:34331 every 8 connection(s)
I20260709 12:14:58.264413  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:14:58.274906 21106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:14:58.281179 21107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:58.281701 21103 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34829
I20260709 12:14:58.282913 21103 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:58.285027 21103 heartbeater.cc:507] Master 127.9.11.126:34829 requested a full tablet report, sending...
W20260709 12:14:58.287457 21109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:14:58.287503  9261 server_base.cc:1061] running on GCE node
I20260709 12:14:58.288492 20901 ts_manager.cc:194] Registered new tserver with Master: 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:14:58.289083  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:14:58.289439  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:14:58.289644  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599298289621 us; error 0 us; skew 500 ppm
I20260709 12:14:58.290458  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:14:58.290609 20901 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39988
I20260709 12:14:58.294054  9261 webserver.cc:533] Webserver started at http://127.9.11.67:41027/ using document root <none> and password file <none>
I20260709 12:14:58.294781  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:14:58.295046  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:14:58.295363  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:14:58.296663  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "58a24ed52e54488ea75011a7096594a7"
format_stamp: "Formatted at 2026-07-09 12:14:58 on dist-test-slave-k289"
I20260709 12:14:58.302175  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:14:58.306381 21114 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.307407  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:14:58.307819  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "58a24ed52e54488ea75011a7096594a7"
format_stamp: "Formatted at 2026-07-09 12:14:58 on dist-test-slave-k289"
I20260709 12:14:58.308122  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:14:58.346202  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:14:58.347854  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:14:58.349643  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:14:58.352311  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:14:58.352528  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.352818  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:14:58.352967  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:14:58.401444  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:34929
I20260709 12:14:58.401587 21176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:34929 every 8 connection(s)
I20260709 12:14:58.418218 21177 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34829
I20260709 12:14:58.418769 21177 heartbeater.cc:461] Registering TS with master...
I20260709 12:14:58.419667 21177 heartbeater.cc:507] Master 127.9.11.126:34829 requested a full tablet report, sending...
I20260709 12:14:58.422341 20901 ts_manager.cc:194] Registered new tserver with Master: 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:58.423163  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017743545s
I20260709 12:14:58.424103 20901 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39998
I20260709 12:14:58.452754 20901 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40004:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:14:58.455461 20901 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:14:58.529204 21142 tablet_service.cc:1511] Processing CreateTablet for tablet 123cc78f793b42c69a6a181cfb626ab4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3b477693dd43434d8981701ac2c4b7fb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:14:58.530930 21141 tablet_service.cc:1511] Processing CreateTablet for tablet 416f5d449d724a27a47b3e21069870a8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3b477693dd43434d8981701ac2c4b7fb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:14:58.531643 20993 tablet_service.cc:1511] Processing CreateTablet for tablet 123cc78f793b42c69a6a181cfb626ab4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3b477693dd43434d8981701ac2c4b7fb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:14:58.532377 21142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 123cc78f793b42c69a6a181cfb626ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.533196 21141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416f5d449d724a27a47b3e21069870a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.533196 20993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 123cc78f793b42c69a6a181cfb626ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.534749 21067 tablet_service.cc:1511] Processing CreateTablet for tablet 416f5d449d724a27a47b3e21069870a8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3b477693dd43434d8981701ac2c4b7fb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:14:58.536202 21067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416f5d449d724a27a47b3e21069870a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.532881 21068 tablet_service.cc:1511] Processing CreateTablet for tablet 123cc78f793b42c69a6a181cfb626ab4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3b477693dd43434d8981701ac2c4b7fb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:14:58.538956 20992 tablet_service.cc:1511] Processing CreateTablet for tablet 416f5d449d724a27a47b3e21069870a8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=3b477693dd43434d8981701ac2c4b7fb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:14:58.540380 20992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 416f5d449d724a27a47b3e21069870a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.538971 21068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 123cc78f793b42c69a6a181cfb626ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.571396 21192 tablet_bootstrap.cc:492] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: Bootstrap starting.
I20260709 12:14:58.581048 21192 tablet_bootstrap.cc:654] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:58.586453 21193 tablet_bootstrap.cc:492] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d: Bootstrap starting.
I20260709 12:14:58.589994 21195 tablet_bootstrap.cc:492] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843: Bootstrap starting.
I20260709 12:14:58.592707 21192 tablet_bootstrap.cc:492] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: No bootstrap required, opened a new log
I20260709 12:14:58.593438 21192 ts_tablet_manager.cc:1403] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.005s
I20260709 12:14:58.595602 21193 tablet_bootstrap.cc:654] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:58.597884 21195 tablet_bootstrap.cc:654] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:58.597512 21192 raft_consensus.cc:359] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.598506 21192 raft_consensus.cc:385] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:58.598851 21192 raft_consensus.cc:740] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Initialized, Role: FOLLOWER
I20260709 12:14:58.599988 21192 consensus_queue.cc:260] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.603653 21177 heartbeater.cc:499] Master 127.9.11.126:34829 was elected leader, sending a full tablet report...
I20260709 12:14:58.604302 21192 ts_tablet_manager.cc:1434] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.005s
I20260709 12:14:58.605927 21192 tablet_bootstrap.cc:492] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: Bootstrap starting.
I20260709 12:14:58.607239 21195 tablet_bootstrap.cc:492] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843: No bootstrap required, opened a new log
I20260709 12:14:58.607913 21195 ts_tablet_manager.cc:1403] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.008s
I20260709 12:14:58.614923 21195 raft_consensus.cc:359] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.615614 21193 tablet_bootstrap.cc:492] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d: No bootstrap required, opened a new log
I20260709 12:14:58.615975 21192 tablet_bootstrap.cc:654] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:58.615810 21195 raft_consensus.cc:385] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:58.616642 21195 raft_consensus.cc:740] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Initialized, Role: FOLLOWER
I20260709 12:14:58.616739 21193 ts_tablet_manager.cc:1403] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.015s
I20260709 12:14:58.617736 21195 consensus_queue.cc:260] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.622320 21193 raft_consensus.cc:359] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.623540 21193 raft_consensus.cc:385] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:58.623983 21193 raft_consensus.cc:740] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Initialized, Role: FOLLOWER
I20260709 12:14:58.625461 21193 consensus_queue.cc:260] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.627905 21195 ts_tablet_manager.cc:1434] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843: Time spent starting tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 12:14:58.628400 21028 heartbeater.cc:499] Master 127.9.11.126:34829 was elected leader, sending a full tablet report...
I20260709 12:14:58.629323 21193 ts_tablet_manager.cc:1434] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:14:58.629576 21195 tablet_bootstrap.cc:492] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843: Bootstrap starting.
I20260709 12:14:58.630007 21192 tablet_bootstrap.cc:492] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: No bootstrap required, opened a new log
I20260709 12:14:58.630219 21103 heartbeater.cc:499] Master 127.9.11.126:34829 was elected leader, sending a full tablet report...
I20260709 12:14:58.630658 21192 ts_tablet_manager.cc:1403] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260709 12:14:58.632623 21193 tablet_bootstrap.cc:492] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d: Bootstrap starting.
I20260709 12:14:58.635308 21192 raft_consensus.cc:359] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.636374 21192 raft_consensus.cc:385] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:58.636718 21192 raft_consensus.cc:740] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Initialized, Role: FOLLOWER
I20260709 12:14:58.637729 21192 consensus_queue.cc:260] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.639118 21195 tablet_bootstrap.cc:654] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:58.640594 21192 ts_tablet_manager.cc:1434] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:14:58.641945 21193 tablet_bootstrap.cc:654] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:58.653226 21195 tablet_bootstrap.cc:492] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843: No bootstrap required, opened a new log
I20260709 12:14:58.653849 21195 ts_tablet_manager.cc:1403] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.007s
I20260709 12:14:58.654014 21193 tablet_bootstrap.cc:492] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d: No bootstrap required, opened a new log
I20260709 12:14:58.654744 21193 ts_tablet_manager.cc:1403] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.003s
I20260709 12:14:58.657572 21195 raft_consensus.cc:359] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.658352 21195 raft_consensus.cc:385] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:58.658149 21193 raft_consensus.cc:359] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.658689 21195 raft_consensus.cc:740] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Initialized, Role: FOLLOWER
I20260709 12:14:58.659053 21193 raft_consensus.cc:385] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:58.659443 21193 raft_consensus.cc:740] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Initialized, Role: FOLLOWER
I20260709 12:14:58.659651 21195 consensus_queue.cc:260] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.660422 21193 consensus_queue.cc:260] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.662168 21195 ts_tablet_manager.cc:1434] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 12:14:58.662817 21193 ts_tablet_manager.cc:1434] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:14:58.677982 21198 raft_consensus.cc:493] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.678628 21198 raft_consensus.cc:515] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.681079 21198 leader_election.cc:290] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:14:58.691843 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843" is_pre_election: true
I20260709 12:14:58.692721 21003 raft_consensus.cc:2468] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 0.
I20260709 12:14:58.693452 21078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e102e6a42244f128030652534b1d52d" is_pre_election: true
I20260709 12:14:58.694223 21117 leader_election.cc:304] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:58.694276 21078 raft_consensus.cc:2468] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 0.
I20260709 12:14:58.695426 21198 raft_consensus.cc:2804] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:58.695870 21198 raft_consensus.cc:493] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:58.696182 21198 raft_consensus.cc:3060] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:58.703379 21198 raft_consensus.cc:515] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.705611 21198 leader_election.cc:290] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 election: Requested vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:14:58.706589 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843"
I20260709 12:14:58.706895 21078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:14:58.707458 21003 raft_consensus.cc:3060] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:58.707644 21078 raft_consensus.cc:3060] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:58.714803 21003 raft_consensus.cc:2468] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 1.
I20260709 12:14:58.715327 21078 raft_consensus.cc:2468] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 1.
I20260709 12:14:58.716220 21117 leader_election.cc:304] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:58.717118 21198 raft_consensus.cc:2804] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:58.718453 21198 raft_consensus.cc:697] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Becoming Leader. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Running, Role: LEADER
I20260709 12:14:58.719358 21198 consensus_queue.cc:237] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:58.729154 20901 catalog_manager.cc:5697] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58a24ed52e54488ea75011a7096594a7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "58a24ed52e54488ea75011a7096594a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:58.847535 21198 raft_consensus.cc:493] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.848423 21198 raft_consensus.cc:515] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.850826 21198 leader_election.cc:290] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:14:58.851867 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843" is_pre_election: true
I20260709 12:14:58.852106 21078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e102e6a42244f128030652534b1d52d" is_pre_election: true
I20260709 12:14:58.852747 21003 raft_consensus.cc:2468] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 0.
I20260709 12:14:58.852855 21078 raft_consensus.cc:2468] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 0.
I20260709 12:14:58.854070 21117 leader_election.cc:304] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:58.854892 21198 raft_consensus.cc:2804] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:58.855316 21198 raft_consensus.cc:493] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:58.855600 21198 raft_consensus.cc:3060] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:58.862111 21198 raft_consensus.cc:515] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.863952 21198 leader_election.cc:290] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 election: Requested vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:14:58.865025 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843"
I20260709 12:14:58.865211 21078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:14:58.865897 21003 raft_consensus.cc:3060] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:58.865900 21078 raft_consensus.cc:3060] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:58.874696 21078 raft_consensus.cc:2468] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 1.
I20260709 12:14:58.874696 21003 raft_consensus.cc:2468] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 1.
I20260709 12:14:58.876314 21117 leader_election.cc:304] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:58.877110 21198 raft_consensus.cc:2804] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:58.877557 21198 raft_consensus.cc:697] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Becoming Leader. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Running, Role: LEADER
I20260709 12:14:58.878325 21198 consensus_queue.cc:237] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:58.886076 20901 catalog_manager.cc:5697] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58a24ed52e54488ea75011a7096594a7 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "58a24ed52e54488ea75011a7096594a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:58.956127 20901 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40018:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:14:58.958706 20901 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:14:58.990752 20993 tablet_service.cc:1511] Processing CreateTablet for tablet 590f7300b25e4e1db9b600717c36e30e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=29756d684d374b9fbbb41e745d4dc7a9]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:14:58.992600 20993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f7300b25e4e1db9b600717c36e30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.992408 21067 tablet_service.cc:1511] Processing CreateTablet for tablet 590f7300b25e4e1db9b600717c36e30e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=29756d684d374b9fbbb41e745d4dc7a9]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:14:58.994102 21067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f7300b25e4e1db9b600717c36e30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.994230 21141 tablet_service.cc:1511] Processing CreateTablet for tablet 590f7300b25e4e1db9b600717c36e30e (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=29756d684d374b9fbbb41e745d4dc7a9]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:14:58.995951 21141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590f7300b25e4e1db9b600717c36e30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:59.000617 21068 tablet_service.cc:1511] Processing CreateTablet for tablet 46301ed239b04992aeda0227682211db (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=29756d684d374b9fbbb41e745d4dc7a9]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:14:59.002198 21068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46301ed239b04992aeda0227682211db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.998920 21142 tablet_service.cc:1511] Processing CreateTablet for tablet 46301ed239b04992aeda0227682211db (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=29756d684d374b9fbbb41e745d4dc7a9]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:14:59.004694 21142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46301ed239b04992aeda0227682211db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:58.998812 20992 tablet_service.cc:1511] Processing CreateTablet for tablet 46301ed239b04992aeda0227682211db (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=29756d684d374b9fbbb41e745d4dc7a9]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:14:59.008474 20992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46301ed239b04992aeda0227682211db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:14:59.014648 21195 tablet_bootstrap.cc:492] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843: Bootstrap starting.
I20260709 12:14:59.018399 21192 tablet_bootstrap.cc:492] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7: Bootstrap starting.
I20260709 12:14:59.021302 21193 tablet_bootstrap.cc:492] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: Bootstrap starting.
I20260709 12:14:59.022783 21195 tablet_bootstrap.cc:654] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:59.025940 21192 tablet_bootstrap.cc:654] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:59.041337 21193 tablet_bootstrap.cc:654] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:59.041980 21192 tablet_bootstrap.cc:492] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7: No bootstrap required, opened a new log
I20260709 12:14:59.042773 21192 ts_tablet_manager.cc:1403] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.007s
I20260709 12:14:59.046545 21192 raft_consensus.cc:359] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.047643 21192 raft_consensus.cc:385] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:59.048100 21192 raft_consensus.cc:740] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Initialized, Role: FOLLOWER
I20260709 12:14:59.049095 21195 tablet_bootstrap.cc:492] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843: No bootstrap required, opened a new log
I20260709 12:14:59.049052 21192 consensus_queue.cc:260] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.049681 21195 ts_tablet_manager.cc:1403] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.005s
I20260709 12:14:59.053159 21195 raft_consensus.cc:359] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.053956 21195 raft_consensus.cc:385] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:59.054298 21195 raft_consensus.cc:740] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Initialized, Role: FOLLOWER
I20260709 12:14:59.055222 21192 ts_tablet_manager.cc:1434] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:14:59.055135 21195 consensus_queue.cc:260] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.058703 21195 ts_tablet_manager.cc:1434] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:14:59.062040 21193 tablet_bootstrap.cc:492] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: No bootstrap required, opened a new log
I20260709 12:14:59.062671 21193 ts_tablet_manager.cc:1403] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.000s
I20260709 12:14:59.066191 21193 raft_consensus.cc:359] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.067052 21193 raft_consensus.cc:385] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:59.067425 21193 raft_consensus.cc:740] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Initialized, Role: FOLLOWER
I20260709 12:14:59.068418 21193 consensus_queue.cc:260] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.073071 21193 ts_tablet_manager.cc:1434] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:14:59.073138 21192 tablet_bootstrap.cc:492] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: Bootstrap starting.
I20260709 12:14:59.073431 21195 tablet_bootstrap.cc:492] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: Bootstrap starting.
I20260709 12:14:59.074429 21193 tablet_bootstrap.cc:492] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: Bootstrap starting.
I20260709 12:14:59.083631 21195 tablet_bootstrap.cc:654] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:59.083781 21193 tablet_bootstrap.cc:654] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:59.083631 21192 tablet_bootstrap.cc:654] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: Neither blocks nor log segments found. Creating new log.
I20260709 12:14:59.090696 21193 tablet_bootstrap.cc:492] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: No bootstrap required, opened a new log
I20260709 12:14:59.097030 21193 ts_tablet_manager.cc:1403] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.001s
I20260709 12:14:59.098852 21195 tablet_bootstrap.cc:492] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: No bootstrap required, opened a new log
I20260709 12:14:59.099661 21192 tablet_bootstrap.cc:492] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: No bootstrap required, opened a new log
I20260709 12:14:59.100328 21192 ts_tablet_manager.cc:1403] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.000s
I20260709 12:14:59.099638 21195 ts_tablet_manager.cc:1403] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.013s
I20260709 12:14:59.100841 21193 raft_consensus.cc:359] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.101809 21193 raft_consensus.cc:385] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:59.102205 21193 raft_consensus.cc:740] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Initialized, Role: FOLLOWER
I20260709 12:14:59.103390 21193 consensus_queue.cc:260] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.104115 21192 raft_consensus.cc:359] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.104382 21195 raft_consensus.cc:359] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.105353 21195 raft_consensus.cc:385] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:59.105475 21192 raft_consensus.cc:385] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:14:59.105706 21195 raft_consensus.cc:740] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Initialized, Role: FOLLOWER
I20260709 12:14:59.105942 21192 raft_consensus.cc:740] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Initialized, Role: FOLLOWER
I20260709 12:14:59.106703 21195 consensus_queue.cc:260] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.106846 21192 consensus_queue.cc:260] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.109709 21195 ts_tablet_manager.cc:1434] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:14:59.108495 21193 ts_tablet_manager.cc:1434] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:14:59.117823 21201 raft_consensus.cc:493] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.118471 21201 raft_consensus.cc:515] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.119860 21192 ts_tablet_manager.cc:1434] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:14:59.124922 21201 leader_election.cc:290] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.130723 21201 raft_consensus.cc:493] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.131374 21201 raft_consensus.cc:515] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.134866 21201 leader_election.cc:290] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.138082 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843" is_pre_election: true
I20260709 12:14:59.139070 21003 raft_consensus.cc:2468] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 0.
I20260709 12:14:59.139328 21200 raft_consensus.cc:493] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.138628 21002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843" is_pre_election: true
I20260709 12:14:59.139971 21200 raft_consensus.cc:515] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.140594 21043 leader_election.cc:304] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 5739a93626c8432aa7a9c5381950f843; no voters: 
I20260709 12:14:59.141667 21201 raft_consensus.cc:2804] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:59.142048 21201 raft_consensus.cc:493] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:59.142416 21201 raft_consensus.cc:3060] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:59.146644 21002 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 0.
I20260709 12:14:59.148650 21200 leader_election.cc:290] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.160084 21043 leader_election.cc:304] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 5739a93626c8432aa7a9c5381950f843; no voters: 
I20260709 12:14:59.170542 21152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a24ed52e54488ea75011a7096594a7" is_pre_election: true
I20260709 12:14:59.171875 21152 raft_consensus.cc:2468] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 0.
I20260709 12:14:59.172514 21151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a24ed52e54488ea75011a7096594a7" is_pre_election: true
I20260709 12:14:59.173403 21151 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 0.
I20260709 12:14:59.176227 21225 raft_consensus.cc:2804] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:59.176963 21225 raft_consensus.cc:493] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:59.177475 21225 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:59.178184 21198 consensus_queue.cc:1048] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:59.176528 21201 raft_consensus.cc:515] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.189409 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843"
I20260709 12:14:59.190133 21151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a24ed52e54488ea75011a7096594a7" is_pre_election: true
I20260709 12:14:59.190660 21078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e102e6a42244f128030652534b1d52d" is_pre_election: true
I20260709 12:14:59.191035 21151 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 0.
I20260709 12:14:59.191522 21201 leader_election.cc:290] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 election: Requested vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.193439 21151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.193529 20966 leader_election.cc:304] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:59.194416 21151 raft_consensus.cc:3060] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:59.192997 21225 raft_consensus.cc:515] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.195245 21200 raft_consensus.cc:2804] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:14:59.195729 21200 raft_consensus.cc:493] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:59.196072 21200 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:59.198462 21225 leader_election.cc:290] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 election: Requested vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.190189 21003 raft_consensus.cc:3060] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:59.199391 21078 raft_consensus.cc:2393] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5739a93626c8432aa7a9c5381950f843 in current term 1: Already voted for candidate 1e102e6a42244f128030652534b1d52d in this term.
I20260709 12:14:59.205883 21151 raft_consensus.cc:2468] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 1.
I20260709 12:14:59.206642 21152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.207119 21002 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5739a93626c8432aa7a9c5381950f843"
I20260709 12:14:59.207443 21152 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:14:59.208431 21041 leader_election.cc:304] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:59.209955 21225 raft_consensus.cc:2804] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:59.210654 21225 raft_consensus.cc:697] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Becoming Leader. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Running, Role: LEADER
I20260709 12:14:59.210860 21200 raft_consensus.cc:515] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.213085 21002 raft_consensus.cc:2393] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1e102e6a42244f128030652534b1d52d in current term 1: Already voted for candidate 5739a93626c8432aa7a9c5381950f843 in this term.
I20260709 12:14:59.212440 21225 consensus_queue.cc:237] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.218165 21078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:14:59.219038 21078 raft_consensus.cc:2393] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5739a93626c8432aa7a9c5381950f843 in current term 1: Already voted for candidate 1e102e6a42244f128030652534b1d52d in this term.
I20260709 12:14:59.222363 21151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.224220 20966 leader_election.cc:304] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843; no voters: 1e102e6a42244f128030652534b1d52d, 58a24ed52e54488ea75011a7096594a7
I20260709 12:14:59.228602 21200 leader_election.cc:290] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 1 election: Requested vote from peers 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.230813 21200 raft_consensus.cc:2749] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:14:59.231876 21003 raft_consensus.cc:2468] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 1.
I20260709 12:14:59.234329 21152 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 1.
I20260709 12:14:59.247663 21041 leader_election.cc:304] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 58a24ed52e54488ea75011a7096594a7; no voters: 5739a93626c8432aa7a9c5381950f843
I20260709 12:14:59.249331 21201 raft_consensus.cc:2804] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:14:59.266834 20900 catalog_manager.cc:5697] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e102e6a42244f128030652534b1d52d (127.9.11.66). New cstate: current_term: 1 leader_uuid: "1e102e6a42244f128030652534b1d52d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.272600 21201 raft_consensus.cc:697] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Becoming Leader. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Running, Role: LEADER
I20260709 12:14:59.274044 21201 consensus_queue.cc:237] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.274641 21198 consensus_queue.cc:1048] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:59.292299 20900 catalog_manager.cc:5697] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e102e6a42244f128030652534b1d52d (127.9.11.66). New cstate: current_term: 1 leader_uuid: "1e102e6a42244f128030652534b1d52d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.343673 21152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4"
dest_uuid: "58a24ed52e54488ea75011a7096594a7"
mode: GRACEFUL
new_leader_uuid: "5739a93626c8432aa7a9c5381950f843"
 from {username='slave'} at 127.0.0.1:54322
I20260709 12:14:59.344398 21152 raft_consensus.cc:606] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Received request to transfer leadership to TS 5739a93626c8432aa7a9c5381950f843
I20260709 12:14:59.350623 21152 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8"
dest_uuid: "58a24ed52e54488ea75011a7096594a7"
mode: GRACEFUL
new_leader_uuid: "1e102e6a42244f128030652534b1d52d"
 from {username='slave'} at 127.0.0.1:54322
I20260709 12:14:59.351297 21152 raft_consensus.cc:606] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Received request to transfer leadership to TS 1e102e6a42244f128030652534b1d52d
I20260709 12:14:59.366885 21078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e"
dest_uuid: "1e102e6a42244f128030652534b1d52d"
mode: GRACEFUL
new_leader_uuid: "5739a93626c8432aa7a9c5381950f843"
 from {username='slave'} at 127.0.0.1:46374
I20260709 12:14:59.367611 21078 raft_consensus.cc:606] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Received request to transfer leadership to TS 5739a93626c8432aa7a9c5381950f843
I20260709 12:14:59.373061 21198 consensus_queue.cc:1048] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:59.374243 21078 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46301ed239b04992aeda0227682211db"
dest_uuid: "1e102e6a42244f128030652534b1d52d"
mode: GRACEFUL
new_leader_uuid: "5739a93626c8432aa7a9c5381950f843"
 from {username='slave'} at 127.0.0.1:46374
I20260709 12:14:59.375072 21078 raft_consensus.cc:606] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Received request to transfer leadership to TS 5739a93626c8432aa7a9c5381950f843
I20260709 12:14:59.386557 21198 consensus_queue.cc:1048] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:14:59.399845 21205 raft_consensus.cc:993] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: : Instructing follower 5739a93626c8432aa7a9c5381950f843 to start an election
I20260709 12:14:59.400450 21198 raft_consensus.cc:1081] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Signalling peer 5739a93626c8432aa7a9c5381950f843 to start an election
I20260709 12:14:59.402300 21002 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4"
dest_uuid: "5739a93626c8432aa7a9c5381950f843"
 from {username='slave'} at 127.0.0.1:55148
I20260709 12:14:59.402874 21002 raft_consensus.cc:493] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:59.403311 21002 raft_consensus.cc:3060] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.412267 21002 raft_consensus.cc:515] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:59.414747 21002 leader_election.cc:290] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 2 election: Requested vote from peers 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929), 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:14:59.415632 21152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.416543 21152 raft_consensus.cc:3055] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:59.416487 21077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "123cc78f793b42c69a6a181cfb626ab4" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:14:59.417217 21152 raft_consensus.cc:740] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Running, Role: LEADER
I20260709 12:14:59.417451 21077 raft_consensus.cc:3060] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.418322 21152 consensus_queue.cc:260] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:59.420009 21152 raft_consensus.cc:3060] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.426563 21077 raft_consensus.cc:2468] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 2.
I20260709 12:14:59.428068 20968 leader_election.cc:304] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 5739a93626c8432aa7a9c5381950f843; no voters: 
I20260709 12:14:59.429126 21230 raft_consensus.cc:2804] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:59.428959 21152 raft_consensus.cc:2468] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 2.
I20260709 12:14:59.429814 21230 raft_consensus.cc:697] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Becoming Leader. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Running, Role: LEADER
I20260709 12:14:59.430786 21230 consensus_queue.cc:237] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } }
I20260709 12:14:59.439273 20901 catalog_manager.cc:5697] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 reported cstate change: term changed from 1 to 2, leader changed from 58a24ed52e54488ea75011a7096594a7 (127.9.11.67) to 5739a93626c8432aa7a9c5381950f843 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "5739a93626c8432aa7a9c5381950f843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.600888 21201 consensus_queue.cc:1048] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:59.605882 21232 consensus_queue.cc:1048] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:59.617354 21225 raft_consensus.cc:993] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: : Instructing follower 5739a93626c8432aa7a9c5381950f843 to start an election
I20260709 12:14:59.617956 21225 raft_consensus.cc:1081] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Signalling peer 5739a93626c8432aa7a9c5381950f843 to start an election
I20260709 12:14:59.620018 21003 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e"
dest_uuid: "5739a93626c8432aa7a9c5381950f843"
 from {username='slave'} at 127.0.0.1:55150
I20260709 12:14:59.621369 21003 raft_consensus.cc:493] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:59.621805 21003 raft_consensus.cc:3060] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.632493 21003 raft_consensus.cc:515] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.635869 21077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:14:59.636686 21077 raft_consensus.cc:3055] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:59.636876 21152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590f7300b25e4e1db9b600717c36e30e" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.637012 21077 raft_consensus.cc:740] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Running, Role: LEADER
I20260709 12:14:59.637687 21152 raft_consensus.cc:3060] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.638350 21077 consensus_queue.cc:260] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.639134 21003 leader_election.cc:290] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 2 election: Requested vote from peers 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.639526 21077 raft_consensus.cc:3060] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.645826 21152 raft_consensus.cc:2468] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 2.
I20260709 12:14:59.646543 21077 raft_consensus.cc:2468] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 2.
I20260709 12:14:59.647253 20966 leader_election.cc:304] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5739a93626c8432aa7a9c5381950f843, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:59.648186 21230 raft_consensus.cc:2804] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:59.648583 21230 raft_consensus.cc:697] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Becoming Leader. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Running, Role: LEADER
I20260709 12:14:59.649328 21230 consensus_queue.cc:237] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.656474 20899 catalog_manager.cc:5697] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 reported cstate change: term changed from 1 to 2, leader changed from 1e102e6a42244f128030652534b1d52d (127.9.11.66) to 5739a93626c8432aa7a9c5381950f843 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "5739a93626c8432aa7a9c5381950f843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.737504 21205 raft_consensus.cc:993] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: : Instructing follower 1e102e6a42244f128030652534b1d52d to start an election
I20260709 12:14:59.738085 21198 raft_consensus.cc:1081] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Signalling peer 1e102e6a42244f128030652534b1d52d to start an election
I20260709 12:14:59.740135 21077 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8"
dest_uuid: "1e102e6a42244f128030652534b1d52d"
 from {username='slave'} at 127.0.0.1:46354
I20260709 12:14:59.740828 21077 raft_consensus.cc:493] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:59.741243 21077 raft_consensus.cc:3060] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.742583 21201 consensus_queue.cc:1048] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:14:59.749621 21077 raft_consensus.cc:515] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.751062 21225 raft_consensus.cc:993] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: : Instructing follower 5739a93626c8432aa7a9c5381950f843 to start an election
I20260709 12:14:59.751624 21225 raft_consensus.cc:1081] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Signalling peer 5739a93626c8432aa7a9c5381950f843 to start an election
I20260709 12:14:59.753101 21077 leader_election.cc:290] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 2 election: Requested vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.754772 21003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5739a93626c8432aa7a9c5381950f843"
I20260709 12:14:59.754978 21001 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46301ed239b04992aeda0227682211db"
dest_uuid: "5739a93626c8432aa7a9c5381950f843"
 from {username='slave'} at 127.0.0.1:55150
I20260709 12:14:59.755532 21003 raft_consensus.cc:3060] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.758127 21001 raft_consensus.cc:493] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:59.758625 21001 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.760246 21152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "416f5d449d724a27a47b3e21069870a8" candidate_uuid: "1e102e6a42244f128030652534b1d52d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.761019 21152 raft_consensus.cc:3055] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:59.761514 21225 consensus_queue.cc:1048] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:14:59.762111 21152 raft_consensus.cc:740] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Running, Role: LEADER
I20260709 12:14:59.763121 21152 consensus_queue.cc:260] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.764390 21152 raft_consensus.cc:3060] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.774912 21152 raft_consensus.cc:2468] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 2.
I20260709 12:14:59.774587 21001 raft_consensus.cc:515] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.776374 21003 raft_consensus.cc:2468] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e102e6a42244f128030652534b1d52d in term 2.
I20260709 12:14:59.776638 21041 leader_election.cc:304] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:14:59.777628 21225 raft_consensus.cc:2804] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:59.778246 21225 raft_consensus.cc:697] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 2 LEADER]: Becoming Leader. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Running, Role: LEADER
I20260709 12:14:59.779211 21225 consensus_queue.cc:237] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.782338 21077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:14:59.782851 21001 leader_election.cc:290] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 2 election: Requested vote from peers 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331), 58a24ed52e54488ea75011a7096594a7 (127.9.11.67:34929)
I20260709 12:14:59.783072 21077 raft_consensus.cc:3055] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:14:59.783459 21077 raft_consensus.cc:740] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1e102e6a42244f128030652534b1d52d, State: Running, Role: LEADER
I20260709 12:14:59.784511 21152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "5739a93626c8432aa7a9c5381950f843" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "58a24ed52e54488ea75011a7096594a7"
I20260709 12:14:59.784375 21077 consensus_queue.cc:260] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.785245 21152 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.785949 21077 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:14:59.790478 20900 catalog_manager.cc:5697] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d reported cstate change: term changed from 1 to 2, leader changed from 58a24ed52e54488ea75011a7096594a7 (127.9.11.67) to 1e102e6a42244f128030652534b1d52d (127.9.11.66). New cstate: current_term: 2 leader_uuid: "1e102e6a42244f128030652534b1d52d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.793763 21077 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 2.
I20260709 12:14:59.795210 21152 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5739a93626c8432aa7a9c5381950f843 in term 2.
I20260709 12:14:59.795197 20968 leader_election.cc:304] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 5739a93626c8432aa7a9c5381950f843; no voters: 
I20260709 12:14:59.796340 21230 raft_consensus.cc:2804] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:14:59.796942 21230 raft_consensus.cc:697] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Becoming Leader. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Running, Role: LEADER
I20260709 12:14:59.798080 21230 consensus_queue.cc:237] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:14:59.805305 20900 catalog_manager.cc:5697] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 reported cstate change: term changed from 1 to 2, leader changed from 1e102e6a42244f128030652534b1d52d (127.9.11.66) to 5739a93626c8432aa7a9c5381950f843 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "5739a93626c8432aa7a9c5381950f843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:59.877491 21152 raft_consensus.cc:1275] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Refusing update from remote peer 5739a93626c8432aa7a9c5381950f843: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:59.877920  9261 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:14:59.879206 21230 consensus_queue.cc:1048] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:59.880107  9261 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:14:59.880698  9261 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:14:59.891739 21077 raft_consensus.cc:1275] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Refusing update from remote peer 5739a93626c8432aa7a9c5381950f843: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:14:59.894207 21230 consensus_queue.cc:1048] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:14:59.900208 21001 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46301ed239b04992aeda0227682211db"
dest_uuid: "5739a93626c8432aa7a9c5381950f843"
mode: GRACEFUL
new_leader_uuid: "58a24ed52e54488ea75011a7096594a7"
 from {username='slave'} at 127.0.0.1:55160
I20260709 12:14:59.900910 21001 raft_consensus.cc:606] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Received request to transfer leadership to TS 58a24ed52e54488ea75011a7096594a7
I20260709 12:14:59.903137  9261 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:15:00.046934 21077 raft_consensus.cc:1275] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Refusing update from remote peer 5739a93626c8432aa7a9c5381950f843: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:00.048738 21230 consensus_queue.cc:1048] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:00.057932 21152 raft_consensus.cc:1275] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Refusing update from remote peer 5739a93626c8432aa7a9c5381950f843: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:00.059899 21230 consensus_queue.cc:1048] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:00.191383 21152 raft_consensus.cc:1275] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Refusing update from remote peer 1e102e6a42244f128030652534b1d52d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:00.192945 21225 consensus_queue.cc:1048] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:00.202374 21001 raft_consensus.cc:1275] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Refusing update from remote peer 1e102e6a42244f128030652534b1d52d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:00.204375 21232 consensus_queue.cc:1048] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:00.312373 21077 raft_consensus.cc:1275] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Refusing update from remote peer 5739a93626c8432aa7a9c5381950f843: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:00.314217 21200 consensus_queue.cc:1048] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:00.319334 21152 raft_consensus.cc:1275] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Refusing update from remote peer 5739a93626c8432aa7a9c5381950f843: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:00.321108 21230 consensus_queue.cc:1048] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:00.331751 21200 raft_consensus.cc:993] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: : Instructing follower 58a24ed52e54488ea75011a7096594a7 to start an election
I20260709 12:15:00.332472 21200 raft_consensus.cc:1081] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Signalling peer 58a24ed52e54488ea75011a7096594a7 to start an election
I20260709 12:15:00.335651 21152 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46301ed239b04992aeda0227682211db"
dest_uuid: "58a24ed52e54488ea75011a7096594a7"
 from {username='slave'} at 127.0.0.1:54312
I20260709 12:15:00.336555 21152 raft_consensus.cc:493] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:00.337009 21152 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:00.345170 21152 raft_consensus.cc:515] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:15:00.347453 21152 leader_election.cc:290] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 3 election: Requested vote from peers 5739a93626c8432aa7a9c5381950f843 (127.9.11.65:35217), 1e102e6a42244f128030652534b1d52d (127.9.11.66:34331)
I20260709 12:15:00.348753 21001 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5739a93626c8432aa7a9c5381950f843"
I20260709 12:15:00.349143 21077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46301ed239b04992aeda0227682211db" candidate_uuid: "58a24ed52e54488ea75011a7096594a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1e102e6a42244f128030652534b1d52d"
I20260709 12:15:00.349632 21001 raft_consensus.cc:3055] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:00.349929 21077 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:00.350123 21001 raft_consensus.cc:740] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5739a93626c8432aa7a9c5381950f843, State: Running, Role: LEADER
I20260709 12:15:00.351282 21001 consensus_queue.cc:260] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:15:00.352977 21001 raft_consensus.cc:3060] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:00.357717 21077 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 3.
I20260709 12:15:00.359342 21117 leader_election.cc:304] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e102e6a42244f128030652534b1d52d, 58a24ed52e54488ea75011a7096594a7; no voters: 
I20260709 12:15:00.360158 21001 raft_consensus.cc:2468] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a24ed52e54488ea75011a7096594a7 in term 3.
I20260709 12:15:00.360419 21198 raft_consensus.cc:2804] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:00.361996 21198 raft_consensus.cc:697] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 3 LEADER]: Becoming Leader. State: Replica: 58a24ed52e54488ea75011a7096594a7, State: Running, Role: LEADER
I20260709 12:15:00.363021 21198 consensus_queue.cc:237] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } }
I20260709 12:15:00.369858 20901 catalog_manager.cc:5697] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 reported cstate change: term changed from 2 to 3, leader changed from 5739a93626c8432aa7a9c5381950f843 (127.9.11.65) to 58a24ed52e54488ea75011a7096594a7 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "58a24ed52e54488ea75011a7096594a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5739a93626c8432aa7a9c5381950f843" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 35217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e102e6a42244f128030652534b1d52d" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 34331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a24ed52e54488ea75011a7096594a7" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 34929 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:00.540021  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
I20260709 12:15:00.564620  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:00.565549  9261 tablet_replica.cc:333] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843: stopping tablet replica
I20260709 12:15:00.566286  9261 raft_consensus.cc:2243] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.566893  9261 raft_consensus.cc:2272] T 46301ed239b04992aeda0227682211db P 5739a93626c8432aa7a9c5381950f843 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.569940  9261 tablet_replica.cc:333] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843: stopping tablet replica
I20260709 12:15:00.570643  9261 raft_consensus.cc:2243] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:00.571741  9261 raft_consensus.cc:2272] T 590f7300b25e4e1db9b600717c36e30e P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.574440  9261 tablet_replica.cc:333] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843: stopping tablet replica
I20260709 12:15:00.575193  9261 raft_consensus.cc:2243] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.575701  9261 raft_consensus.cc:2272] T 416f5d449d724a27a47b3e21069870a8 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.578190  9261 tablet_replica.cc:333] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843: stopping tablet replica
I20260709 12:15:00.578747  9261 raft_consensus.cc:2243] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:00.579753  9261 raft_consensus.cc:2272] T 123cc78f793b42c69a6a181cfb626ab4 P 5739a93626c8432aa7a9c5381950f843 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.596374  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
I20260709 12:15:00.612254  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
I20260709 12:15:00.634393  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:00.635193  9261 tablet_replica.cc:333] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d: stopping tablet replica
I20260709 12:15:00.635854  9261 raft_consensus.cc:2243] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.636368  9261 raft_consensus.cc:2272] T 590f7300b25e4e1db9b600717c36e30e P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.639117  9261 tablet_replica.cc:333] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d: stopping tablet replica
I20260709 12:15:00.639719  9261 raft_consensus.cc:2243] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.640182  9261 raft_consensus.cc:2272] T 46301ed239b04992aeda0227682211db P 1e102e6a42244f128030652534b1d52d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.642971  9261 tablet_replica.cc:333] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d: stopping tablet replica
I20260709 12:15:00.643594  9261 raft_consensus.cc:2243] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.644038  9261 raft_consensus.cc:2272] T 123cc78f793b42c69a6a181cfb626ab4 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.646628  9261 tablet_replica.cc:333] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d: stopping tablet replica
I20260709 12:15:00.647215  9261 raft_consensus.cc:2243] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:00.648243  9261 raft_consensus.cc:2272] T 416f5d449d724a27a47b3e21069870a8 P 1e102e6a42244f128030652534b1d52d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.673954  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
I20260709 12:15:00.690343  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
I20260709 12:15:00.713434  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:00.714190  9261 tablet_replica.cc:333] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: stopping tablet replica
I20260709 12:15:00.714908  9261 raft_consensus.cc:2243] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:00.715828  9261 pending_rounds.cc:70] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: Trying to abort 1 pending ops.
I20260709 12:15:00.716064  9261 pending_rounds.cc:77] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305622734295527424 op_type: NO_OP noop_request { }
I20260709 12:15:00.716424  9261 raft_consensus.cc:2889] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:15:00.716753  9261 raft_consensus.cc:2272] T 46301ed239b04992aeda0227682211db P 58a24ed52e54488ea75011a7096594a7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.719625  9261 tablet_replica.cc:333] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7: stopping tablet replica
I20260709 12:15:00.720274  9261 raft_consensus.cc:2243] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.720799  9261 raft_consensus.cc:2272] T 590f7300b25e4e1db9b600717c36e30e P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.724074  9261 tablet_replica.cc:333] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7: stopping tablet replica
I20260709 12:15:00.724707  9261 raft_consensus.cc:2243] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.725232  9261 raft_consensus.cc:2272] T 123cc78f793b42c69a6a181cfb626ab4 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.728117  9261 tablet_replica.cc:333] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7: stopping tablet replica
I20260709 12:15:00.728629  9261 raft_consensus.cc:2243] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:00.729041  9261 raft_consensus.cc:2272] T 416f5d449d724a27a47b3e21069870a8 P 58a24ed52e54488ea75011a7096594a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.744710  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:15:00.761272  9261 master.cc:562] Master@127.9.11.126:34829 shutting down...
I20260709 12:15:00.780100  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:00.780836  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:00.781241  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P c4fa74e589564011aacc08dd74e7dbd7: stopping tablet replica
I20260709 12:15:00.801921  9261 master.cc:584] Master@127.9.11.126:34829 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3083 ms)
[----------] 6 tests from LeaderRebalancerTest (406628 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:15:00.837894  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:37351
I20260709 12:15:00.839753  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:00.845566 21252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:00.846320 21253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:00.849190 21255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:00.849782  9261 server_base.cc:1061] running on GCE node
I20260709 12:15:00.850785  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:00.851086  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:00.851248  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599300851214 us; error 0 us; skew 500 ppm
I20260709 12:15:00.851867  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:00.854429  9261 webserver.cc:533] Webserver started at http://127.9.11.126:33327/ using document root <none> and password file <none>
I20260709 12:15:00.855033  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:00.855257  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:00.855571  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:00.856820  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "8a70a01ac6a445b7942a6858b54cbed6"
format_stamp: "Formatted at 2026-07-09 12:15:00 on dist-test-slave-k289"
I20260709 12:15:00.861665  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:15:00.865549 21260 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:00.866365  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:15:00.866670  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "8a70a01ac6a445b7942a6858b54cbed6"
format_stamp: "Formatted at 2026-07-09 12:15:00 on dist-test-slave-k289"
I20260709 12:15:00.866969  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:00.889843  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:00.891695  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:00.941154  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:37351
I20260709 12:15:00.941309 21311 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:37351 every 8 connection(s)
I20260709 12:15:00.945827 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:00.958393 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6: Bootstrap starting.
I20260709 12:15:00.964071 21312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:00.968636 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6: No bootstrap required, opened a new log
I20260709 12:15:00.971150 21312 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a70a01ac6a445b7942a6858b54cbed6" member_type: VOTER }
I20260709 12:15:00.971575 21312 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:00.971791 21312 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a70a01ac6a445b7942a6858b54cbed6, State: Initialized, Role: FOLLOWER
I20260709 12:15:00.972353 21312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a70a01ac6a445b7942a6858b54cbed6" member_type: VOTER }
I20260709 12:15:00.972793 21312 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:00.972990 21312 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:00.973206 21312 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:00.979177 21312 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a70a01ac6a445b7942a6858b54cbed6" member_type: VOTER }
I20260709 12:15:00.979779 21312 leader_election.cc:304] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a70a01ac6a445b7942a6858b54cbed6; no voters: 
I20260709 12:15:00.981084 21312 leader_election.cc:290] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:00.981519 21315 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:00.983129 21315 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 1 LEADER]: Becoming Leader. State: Replica: 8a70a01ac6a445b7942a6858b54cbed6, State: Running, Role: LEADER
I20260709 12:15:00.983909 21315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a70a01ac6a445b7942a6858b54cbed6" member_type: VOTER }
I20260709 12:15:00.984444 21312 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:00.987269 21316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a70a01ac6a445b7942a6858b54cbed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a70a01ac6a445b7942a6858b54cbed6" member_type: VOTER } }
I20260709 12:15:00.987286 21317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a70a01ac6a445b7942a6858b54cbed6. Latest consensus state: current_term: 1 leader_uuid: "8a70a01ac6a445b7942a6858b54cbed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a70a01ac6a445b7942a6858b54cbed6" member_type: VOTER } }
I20260709 12:15:00.988229 21317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:00.989614 21316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:00.991485 21321 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:01.000137 21321 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:01.002916  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:01.011112 21321 catalog_manager.cc:1383] Generated new cluster ID: 0a6fd77491354cc9a06ea168e7aab253
I20260709 12:15:01.011523 21321 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:01.029599 21321 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:01.031311 21321 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:01.046336 21321 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6: Generated new TSK 0
I20260709 12:15:01.047211 21321 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:01.070513  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:01.078094 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:01.079767 21334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:01.080880 21336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:01.081873  9261 server_base.cc:1061] running on GCE node
I20260709 12:15:01.082883  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:01.083146  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:01.083280  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599301083268 us; error 0 us; skew 500 ppm
I20260709 12:15:01.083847  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:01.086410  9261 webserver.cc:533] Webserver started at http://127.9.11.65:36593/ using document root <none> and password file <none>
I20260709 12:15:01.086959  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:01.087263  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:01.087584  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:01.088919  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "ef39358b552d46568d4673bc03cb2e12"
format_stamp: "Formatted at 2026-07-09 12:15:01 on dist-test-slave-k289"
I20260709 12:15:01.093912  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:15:01.097886 21341 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.098701  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:15:01.099095  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "ef39358b552d46568d4673bc03cb2e12"
format_stamp: "Formatted at 2026-07-09 12:15:01 on dist-test-slave-k289"
I20260709 12:15:01.099438  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:01.119230  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:01.120648  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:01.122362  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:01.125082  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:01.125329  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.125590  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:01.125771  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.173002  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:40483
I20260709 12:15:01.173125 21403 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:40483 every 8 connection(s)
I20260709 12:15:01.178864  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:01.186761 21408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:01.188215 21409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:01.192140  9261 server_base.cc:1061] running on GCE node
W20260709 12:15:01.192241 21411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:01.192698 21404 heartbeater.cc:344] Connected to a master server at 127.9.11.126:37351
I20260709 12:15:01.193161 21404 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:01.194087 21404 heartbeater.cc:507] Master 127.9.11.126:37351 requested a full tablet report, sending...
I20260709 12:15:01.194480  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:01.194779  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:01.194974  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599301194951 us; error 0 us; skew 500 ppm
I20260709 12:15:01.195997  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:01.196764 21277 ts_manager.cc:194] Registered new tserver with Master: ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483)
I20260709 12:15:01.198911  9261 webserver.cc:533] Webserver started at http://127.9.11.66:36705/ using document root <none> and password file <none>
I20260709 12:15:01.199420 21277 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59260
I20260709 12:15:01.199597  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:01.199962  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:01.200251  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:01.201831  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "aa209a8923e14e68b1ba654178cedee3"
format_stamp: "Formatted at 2026-07-09 12:15:01 on dist-test-slave-k289"
I20260709 12:15:01.206811  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.006s
I20260709 12:15:01.210564 21416 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.211364  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:15:01.211723  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "aa209a8923e14e68b1ba654178cedee3"
format_stamp: "Formatted at 2026-07-09 12:15:01 on dist-test-slave-k289"
I20260709 12:15:01.212075  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:01.224654  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:01.225888  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:01.227540  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:01.230086  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:01.230341  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.230597  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:01.230788  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.279284  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:46681
I20260709 12:15:01.279417 21478 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:46681 every 8 connection(s)
I20260709 12:15:01.285153  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:01.295066 21482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:01.295269 21483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:01.298921 21479 heartbeater.cc:344] Connected to a master server at 127.9.11.126:37351
W20260709 12:15:01.298831 21485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:01.299650  9261 server_base.cc:1061] running on GCE node
I20260709 12:15:01.299768 21479 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:01.300935  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:15:01.301070 21479 heartbeater.cc:507] Master 127.9.11.126:37351 requested a full tablet report, sending...
W20260709 12:15:01.301175  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:01.301671  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599301301649 us; error 0 us; skew 500 ppm
I20260709 12:15:01.302392  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:01.303751 21277 ts_manager.cc:194] Registered new tserver with Master: aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:01.305483 21277 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59274
I20260709 12:15:01.305626  9261 webserver.cc:533] Webserver started at http://127.9.11.67:33555/ using document root <none> and password file <none>
I20260709 12:15:01.306294  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:01.306535  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:01.306839  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:01.308188  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "a9ed2c75babe4e49a17ab052415d1434"
format_stamp: "Formatted at 2026-07-09 12:15:01 on dist-test-slave-k289"
I20260709 12:15:01.313771  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:15:01.317827 21490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.318739  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:15:01.319192  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "a9ed2c75babe4e49a17ab052415d1434"
format_stamp: "Formatted at 2026-07-09 12:15:01 on dist-test-slave-k289"
I20260709 12:15:01.319545  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:01.340899  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:01.342319  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:01.344077  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:01.346709  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:01.346968  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.347256  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:01.347453  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:01.394820  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:35351
I20260709 12:15:01.395046 21552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:35351 every 8 connection(s)
I20260709 12:15:01.411216 21553 heartbeater.cc:344] Connected to a master server at 127.9.11.126:37351
I20260709 12:15:01.411871 21553 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:01.413057 21553 heartbeater.cc:507] Master 127.9.11.126:37351 requested a full tablet report, sending...
I20260709 12:15:01.415627 21277 ts_manager.cc:194] Registered new tserver with Master: a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:01.416078  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017405251s
I20260709 12:15:01.417543 21277 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59290
I20260709 12:15:01.444984 21277 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59302:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:01.447932 21277 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:01.518849 21444 tablet_service.cc:1511] Processing CreateTablet for tablet de0f5c45176b4a82a53549389e30b72e (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:01.521430 21444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0f5c45176b4a82a53549389e30b72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.522570 21443 tablet_service.cc:1511] Processing CreateTablet for tablet d4f9a1ecaab34de6ade6f289f05c5985 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:01.525367 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f9a1ecaab34de6ade6f289f05c5985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.527292 21369 tablet_service.cc:1511] Processing CreateTablet for tablet de0f5c45176b4a82a53549389e30b72e (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:01.527694 21368 tablet_service.cc:1511] Processing CreateTablet for tablet d4f9a1ecaab34de6ade6f289f05c5985 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:01.529003 21369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0f5c45176b4a82a53549389e30b72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.531158 21367 tablet_service.cc:1511] Processing CreateTablet for tablet 88e983ca86ec4bf795b7e5fc727b9def (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:01.532681 21367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e983ca86ec4bf795b7e5fc727b9def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.535427 21442 tablet_service.cc:1511] Processing CreateTablet for tablet 88e983ca86ec4bf795b7e5fc727b9def (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:01.536993 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e983ca86ec4bf795b7e5fc727b9def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.541975 21441 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca48b558d1c438c868aa42cdff43269 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:01.543761 21441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca48b558d1c438c868aa42cdff43269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.548209 21440 tablet_service.cc:1511] Processing CreateTablet for tablet 80a0c0185f6e4e9193531352768d711a (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:01.549901 21440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80a0c0185f6e4e9193531352768d711a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.553229 21517 tablet_service.cc:1511] Processing CreateTablet for tablet d4f9a1ecaab34de6ade6f289f05c5985 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:01.554177 21366 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca48b558d1c438c868aa42cdff43269 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:01.554930 21517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f9a1ecaab34de6ade6f289f05c5985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.558424 21512 tablet_service.cc:1511] Processing CreateTablet for tablet 002b11df78754235ae83eb63aaebe2c9 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:01.559988 21512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002b11df78754235ae83eb63aaebe2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.560210 21439 tablet_service.cc:1511] Processing CreateTablet for tablet d90860574723485e8d764982f9abbf13 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:01.561772 21439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d90860574723485e8d764982f9abbf13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.554963 21516 tablet_service.cc:1511] Processing CreateTablet for tablet 88e983ca86ec4bf795b7e5fc727b9def (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:01.564127 21438 tablet_service.cc:1511] Processing CreateTablet for tablet 002b11df78754235ae83eb63aaebe2c9 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:01.564591 21516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e983ca86ec4bf795b7e5fc727b9def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.565411 21438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002b11df78754235ae83eb63aaebe2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.556067 21515 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca48b558d1c438c868aa42cdff43269 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:01.568596 21515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca48b558d1c438c868aa42cdff43269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.571306 21511 tablet_service.cc:1511] Processing CreateTablet for tablet 75eb7b8d72c24d7f8950630e908abf8c (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:01.572511 21361 tablet_service.cc:1511] Processing CreateTablet for tablet 93d74552227943f4afdee2a1ea3c547d (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:01.574370 21437 tablet_service.cc:1511] Processing CreateTablet for tablet 75eb7b8d72c24d7f8950630e908abf8c (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:01.575943 21437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75eb7b8d72c24d7f8950630e908abf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.580325 21510 tablet_service.cc:1511] Processing CreateTablet for tablet 93d74552227943f4afdee2a1ea3c547d (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:01.583021 21364 tablet_service.cc:1511] Processing CreateTablet for tablet d90860574723485e8d764982f9abbf13 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:01.585793 21363 tablet_service.cc:1511] Processing CreateTablet for tablet 002b11df78754235ae83eb63aaebe2c9 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:01.588261 21436 tablet_service.cc:1511] Processing CreateTablet for tablet 93d74552227943f4afdee2a1ea3c547d (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:01.589893 21436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d74552227943f4afdee2a1ea3c547d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.572971 21511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75eb7b8d72c24d7f8950630e908abf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.557868 21514 tablet_service.cc:1511] Processing CreateTablet for tablet 80a0c0185f6e4e9193531352768d711a (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:01.592487 21514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80a0c0185f6e4e9193531352768d711a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.595341 21510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d74552227943f4afdee2a1ea3c547d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.571306 21362 tablet_service.cc:1511] Processing CreateTablet for tablet 75eb7b8d72c24d7f8950630e908abf8c (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:01.557221 21513 tablet_service.cc:1511] Processing CreateTablet for tablet d90860574723485e8d764982f9abbf13 (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:01.556067 21518 tablet_service.cc:1511] Processing CreateTablet for tablet de0f5c45176b4a82a53549389e30b72e (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:01.556608 21365 tablet_service.cc:1511] Processing CreateTablet for tablet 80a0c0185f6e4e9193531352768d711a (DEFAULT_TABLE table=test-workload [id=2d69581ba6a4475fb5da187b9abc5e58]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:01.557979 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f9a1ecaab34de6ade6f289f05c5985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.602624 21363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002b11df78754235ae83eb63aaebe2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.603442 21513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d90860574723485e8d764982f9abbf13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.606648 21362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75eb7b8d72c24d7f8950630e908abf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.607314 21518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0f5c45176b4a82a53549389e30b72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.609983 21365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80a0c0185f6e4e9193531352768d711a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.612749 21364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d90860574723485e8d764982f9abbf13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.627521 21361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d74552227943f4afdee2a1ea3c547d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.631451 21366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca48b558d1c438c868aa42cdff43269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:01.672420 21568 tablet_bootstrap.cc:492] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:01.706163 21568 tablet_bootstrap.cc:654] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.738883 21569 tablet_bootstrap.cc:492] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:01.754663 21570 tablet_bootstrap.cc:492] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:01.763353 21570 tablet_bootstrap.cc:654] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.767282 21569 tablet_bootstrap.cc:654] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.777148 21568 tablet_bootstrap.cc:492] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:01.777962 21568 ts_tablet_manager.cc:1403] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.106s	user 0.014s	sys 0.000s
I20260709 12:15:01.787403 21568 raft_consensus.cc:359] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.788326 21568 raft_consensus.cc:385] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.788666 21568 raft_consensus.cc:740] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.789971 21568 consensus_queue.cc:260] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.800741 21568 ts_tablet_manager.cc:1434] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:15:01.801741 21479 heartbeater.cc:499] Master 127.9.11.126:37351 was elected leader, sending a full tablet report...
I20260709 12:15:01.806735 21568 tablet_bootstrap.cc:492] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:01.809840 21569 tablet_bootstrap.cc:492] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:01.810009 21570 tablet_bootstrap.cc:492] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:01.810900 21570 ts_tablet_manager.cc:1403] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.023s
I20260709 12:15:01.814090 21568 tablet_bootstrap.cc:654] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.814462 21570 raft_consensus.cc:359] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.815389 21570 raft_consensus.cc:385] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.815701 21570 raft_consensus.cc:740] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.816512 21570 consensus_queue.cc:260] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.818199 21569 ts_tablet_manager.cc:1403] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.022s
I20260709 12:15:01.821626 21569 raft_consensus.cc:359] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.822438 21569 raft_consensus.cc:385] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.822721 21569 raft_consensus.cc:740] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.823520 21569 consensus_queue.cc:260] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.827060 21553 heartbeater.cc:499] Master 127.9.11.126:37351 was elected leader, sending a full tablet report...
I20260709 12:15:01.827251 21569 ts_tablet_manager.cc:1434] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:15:01.832823 21568 tablet_bootstrap.cc:492] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:01.833550 21568 ts_tablet_manager.cc:1403] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.008s
I20260709 12:15:01.833877 21569 tablet_bootstrap.cc:492] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:01.837246 21568 raft_consensus.cc:359] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.838213 21568 raft_consensus.cc:385] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.838573 21568 raft_consensus.cc:740] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.840623 21404 heartbeater.cc:499] Master 127.9.11.126:37351 was elected leader, sending a full tablet report...
I20260709 12:15:01.841190 21570 ts_tablet_manager.cc:1434] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.011s
I20260709 12:15:01.842483 21570 tablet_bootstrap.cc:492] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:01.843042 21569 tablet_bootstrap.cc:654] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.847107 21568 consensus_queue.cc:260] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.849844 21570 tablet_bootstrap.cc:654] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.853325 21568 ts_tablet_manager.cc:1434] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.003s
I20260709 12:15:01.854434 21576 raft_consensus.cc:493] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.855234 21576 raft_consensus.cc:515] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.862342 21570 tablet_bootstrap.cc:492] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:01.863018 21570 ts_tablet_manager.cc:1403] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.006s
I20260709 12:15:01.866338 21570 raft_consensus.cc:359] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.867081 21570 raft_consensus.cc:385] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.867373 21570 raft_consensus.cc:740] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.868433 21570 consensus_queue.cc:260] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.871671 21576 leader_election.cc:290] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:01.878342 21570 ts_tablet_manager.cc:1434] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.011s
I20260709 12:15:01.879520 21570 tablet_bootstrap.cc:492] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:01.880927 21568 tablet_bootstrap.cc:492] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:01.882920 21577 raft_consensus.cc:493] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.883656 21577 raft_consensus.cc:515] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.886950 21570 tablet_bootstrap.cc:654] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.911531 21568 tablet_bootstrap.cc:654] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:01.931331 21577 leader_election.cc:290] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:01.934772 21582 raft_consensus.cc:493] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.935616 21582 raft_consensus.cc:515] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.946008 21582 leader_election.cc:290] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:01.952548 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
W20260709 12:15:01.956598 21494 leader_election.cc:343] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:01.959457 21568 tablet_bootstrap.cc:492] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:01.960070 21569 tablet_bootstrap.cc:492] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:01.960831 21569 ts_tablet_manager.cc:1403] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.127s	user 0.051s	sys 0.051s
I20260709 12:15:01.964331 21570 tablet_bootstrap.cc:492] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:01.965039 21570 ts_tablet_manager.cc:1403] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.086s	user 0.038s	sys 0.032s
I20260709 12:15:01.964608 21569 raft_consensus.cc:359] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.965751 21569 raft_consensus.cc:385] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.966156 21569 raft_consensus.cc:740] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.965890 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d90860574723485e8d764982f9abbf13" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:01.967554 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88e983ca86ec4bf795b7e5fc727b9def" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:01.968338 21574 raft_consensus.cc:493] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.968499 21569 consensus_queue.cc:260] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.969278 21574 raft_consensus.cc:515] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
W20260709 12:15:01.969611 21344 leader_election.cc:343] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:01.970496 21344 leader_election.cc:343] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:01.968683 21570 raft_consensus.cc:359] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.978384 21570 raft_consensus.cc:385] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:01.978721 21574 leader_election.cc:290] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:01.978834 21570 raft_consensus.cc:740] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:01.980556 21569 ts_tablet_manager.cc:1434] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.005s
I20260709 12:15:01.980518 21570 consensus_queue.cc:260] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:01.982357 21569 tablet_bootstrap.cc:492] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:01.988026 21570 ts_tablet_manager.cc:1434] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 12:15:01.989985 21570 tablet_bootstrap.cc:492] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:01.995303 21568 ts_tablet_manager.cc:1403] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.115s	user 0.022s	sys 0.002s
W20260709 12:15:01.997171 21325 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:01.999675 21570 tablet_bootstrap.cc:654] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.003790 21569 tablet_bootstrap.cc:654] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.016028 21569 tablet_bootstrap.cc:492] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.016786 21569 ts_tablet_manager.cc:1403] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.004s
I20260709 12:15:02.020244 21569 raft_consensus.cc:359] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.021438 21569 raft_consensus.cc:385] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.021281 21568 raft_consensus.cc:359] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.021996 21569 raft_consensus.cc:740] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.022358 21568 raft_consensus.cc:385] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.023388 21569 consensus_queue.cc:260] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.026393 21569 ts_tablet_manager.cc:1434] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:15:02.027611 21569 tablet_bootstrap.cc:492] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:02.034662 21569 tablet_bootstrap.cc:654] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.046823 21569 tablet_bootstrap.cc:492] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.022758 21568 raft_consensus.cc:740] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.047291 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d90860574723485e8d764982f9abbf13" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.047632 21569 ts_tablet_manager.cc:1403] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 12:15:02.047609 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88e983ca86ec4bf795b7e5fc727b9def" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.048857 21568 consensus_queue.cc:260] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
W20260709 12:15:02.049702 21342 leader_election.cc:343] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.050309 21342 leader_election.cc:304] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef39358b552d46568d4673bc03cb2e12; no voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:02.051704 21569 raft_consensus.cc:359] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.052758 21569 raft_consensus.cc:385] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.053094 21569 raft_consensus.cc:740] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.051698 21582 raft_consensus.cc:2749] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:02.054049 21569 consensus_queue.cc:260] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.055205 21527 raft_consensus.cc:2468] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:02.056640 21569 ts_tablet_manager.cc:1434] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:15:02.057061 21342 leader_election.cc:304] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:02.057636 21568 ts_tablet_manager.cc:1434] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.058s	user 0.009s	sys 0.000s
I20260709 12:15:02.057832 21569 tablet_bootstrap.cc:492] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:02.058457 21582 raft_consensus.cc:2804] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.058902 21582 raft_consensus.cc:493] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.058974 21568 tablet_bootstrap.cc:492] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:02.059373 21582 raft_consensus.cc:3060] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.061563 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.062341 21453 raft_consensus.cc:2468] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:02.063833 21493 leader_election.cc:304] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.064836 21577 raft_consensus.cc:2804] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.065230 21577 raft_consensus.cc:493] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.065573 21577 raft_consensus.cc:3060] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.074688 21569 tablet_bootstrap.cc:654] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.077764 21570 tablet_bootstrap.cc:492] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:02.078516 21570 ts_tablet_manager.cc:1403] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.089s	user 0.029s	sys 0.025s
I20260709 12:15:02.081691 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
W20260709 12:15:02.083823 21420 leader_election.cc:343] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.085426 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.087422 21570 raft_consensus.cc:359] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.088442 21570 raft_consensus.cc:385] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.088744 21570 raft_consensus.cc:740] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.089637 21570 consensus_queue.cc:260] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.089592 21582 raft_consensus.cc:515] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.093042 21570 ts_tablet_manager.cc:1434] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:15:02.094308 21570 tablet_bootstrap.cc:492] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:02.094450 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88e983ca86ec4bf795b7e5fc727b9def" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
W20260709 12:15:02.096956 21344 leader_election.cc:343] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.099239 21582 leader_election.cc:290] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.102914 21570 tablet_bootstrap.cc:654] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.109089 21577 raft_consensus.cc:515] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.110448 21568 tablet_bootstrap.cc:654] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.111526 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88e983ca86ec4bf795b7e5fc727b9def" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.113847 21528 raft_consensus.cc:3060] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.115547 21570 tablet_bootstrap.cc:492] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:02.116461 21570 ts_tablet_manager.cc:1403] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 12:15:02.113099 21527 raft_consensus.cc:2393] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa209a8923e14e68b1ba654178cedee3 in current term 1: Already voted for candidate a9ed2c75babe4e49a17ab052415d1434 in this term.
I20260709 12:15:02.119313 21417 leader_election.cc:304] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3; no voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.120277 21574 raft_consensus.cc:3060] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.121068 21570 raft_consensus.cc:359] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.122350 21570 raft_consensus.cc:385] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.122732 21570 raft_consensus.cc:740] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.123853 21570 consensus_queue.cc:260] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.128584 21574 raft_consensus.cc:2749] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:02.131978 21570 ts_tablet_manager.cc:1434] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:15:02.133754 21570 tablet_bootstrap.cc:492] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:02.135730 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.136440 21379 raft_consensus.cc:3060] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.145925 21379 raft_consensus.cc:2468] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:02.145998 21570 tablet_bootstrap.cc:654] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.147957 21528 raft_consensus.cc:2468] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:02.148612 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.152040 21568 tablet_bootstrap.cc:492] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:02.152868 21568 ts_tablet_manager.cc:1403] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.004s
I20260709 12:15:02.156929 21568 raft_consensus.cc:359] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.157850 21568 raft_consensus.cc:385] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.158245 21568 raft_consensus.cc:740] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.159188 21568 consensus_queue.cc:260] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.162251 21568 ts_tablet_manager.cc:1434] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:15:02.163552 21568 tablet_bootstrap.cc:492] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:02.164336 21453 raft_consensus.cc:2468] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:02.149413 21577 leader_election.cc:290] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:02.149163 21494 leader_election.cc:304] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.168102 21577 raft_consensus.cc:2804] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.169538 21577 raft_consensus.cc:697] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:02.172247 21568 tablet_bootstrap.cc:654] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.173187 21570 tablet_bootstrap.cc:492] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:02.174105 21570 ts_tablet_manager.cc:1403] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.016s
I20260709 12:15:02.181178 21577 consensus_queue.cc:237] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.178838 21570 raft_consensus.cc:359] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.183557 21570 raft_consensus.cc:385] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.183986 21570 raft_consensus.cc:740] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.185011 21570 consensus_queue.cc:260] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.187095 21568 tablet_bootstrap.cc:492] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:02.187781 21568 ts_tablet_manager.cc:1403] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 12:15:02.188643 21570 ts_tablet_manager.cc:1434] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 12:15:02.189926 21570 tablet_bootstrap.cc:492] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:02.191296 21568 raft_consensus.cc:359] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.192150 21568 raft_consensus.cc:385] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.192483 21568 raft_consensus.cc:740] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
W20260709 12:15:02.193485 21494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:40483, user_credentials={real_user=slave}} blocked reactor thread for 46012.3us
I20260709 12:15:02.193377 21568 consensus_queue.cc:260] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.196564 21568 ts_tablet_manager.cc:1434] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:15:02.197727 21568 tablet_bootstrap.cc:492] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:02.198424 21342 leader_election.cc:304] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:02.199986 21582 raft_consensus.cc:2804] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.200542 21582 raft_consensus.cc:697] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:02.201511 21582 consensus_queue.cc:237] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.205379 21568 tablet_bootstrap.cc:654] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.205236 21574 raft_consensus.cc:493] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.206110 21574 raft_consensus.cc:515] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.206669 21570 tablet_bootstrap.cc:654] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.210217 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.210477 21574 leader_election.cc:290] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
W20260709 12:15:02.211966 21420 leader_election.cc:343] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.216584 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.217383 21528 raft_consensus.cc:2468] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:02.218946 21417 leader_election.cc:304] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.220927 21574 raft_consensus.cc:2804] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.221495 21574 raft_consensus.cc:493] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.221997 21574 raft_consensus.cc:3060] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.231173 21569 tablet_bootstrap.cc:492] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.232260 21569 ts_tablet_manager.cc:1403] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.175s	user 0.064s	sys 0.056s
I20260709 12:15:02.232990 21597 raft_consensus.cc:493] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.237314 21569 raft_consensus.cc:359] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.255486 21569 raft_consensus.cc:385] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.246066 21274 catalog_manager.cc:5697] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.255522 21597 raft_consensus.cc:515] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.256040 21569 raft_consensus.cc:740] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.260172 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.260557 21569 consensus_queue.cc:260] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.263476 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.263624 21569 ts_tablet_manager.cc:1434] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.002s
W20260709 12:15:02.264709 21494 leader_election.cc:343] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.265700 21569 tablet_bootstrap.cc:492] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:02.270431 21604 raft_consensus.cc:493] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.271518 21604 raft_consensus.cc:515] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.260504 21597 leader_election.cc:290] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:02.276501 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.277431 21379 raft_consensus.cc:2468] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:02.279227 21420 leader_election.cc:304] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.281524 21570 tablet_bootstrap.cc:492] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:02.285562 21568 tablet_bootstrap.cc:492] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:02.291726 21568 ts_tablet_manager.cc:1403] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.094s	user 0.032s	sys 0.029s
I20260709 12:15:02.291915 21570 ts_tablet_manager.cc:1403] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.102s	user 0.030s	sys 0.045s
I20260709 12:15:02.293115 21607 raft_consensus.cc:2804] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.285462 21574 raft_consensus.cc:515] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.293694 21607 raft_consensus.cc:493] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.294118 21607 raft_consensus.cc:3060] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.294433 21453 raft_consensus.cc:2393] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9ed2c75babe4e49a17ab052415d1434 in current term 1: Already voted for candidate aa209a8923e14e68b1ba654178cedee3 in this term.
I20260709 12:15:02.296268 21570 raft_consensus.cc:359] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.296084 21568 raft_consensus.cc:359] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.296872 21493 leader_election.cc:304] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434; no voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.297320 21570 raft_consensus.cc:385] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.297401 21568 raft_consensus.cc:385] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.297988 21570 raft_consensus.cc:740] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.298060 21568 raft_consensus.cc:740] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.298703 21577 raft_consensus.cc:3060] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.299206 21570 consensus_queue.cc:260] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.299453 21568 consensus_queue.cc:260] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.296006 21608 raft_consensus.cc:493] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.301944 21608 raft_consensus.cc:515] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.302806 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.303526 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.304178 21379 raft_consensus.cc:3060] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:02.304909 21417 leader_election.cc:343] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.305240 21608 leader_election.cc:290] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.309917 21597 raft_consensus.cc:493] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.310324 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.309405 21574 leader_election.cc:290] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.307503 21604 leader_election.cc:290] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.313113 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.314026 21569 tablet_bootstrap.cc:654] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.315593 21570 ts_tablet_manager.cc:1434] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:15:02.315706 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.320621 21378 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:02.310720 21597 raft_consensus.cc:515] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.322098 21570 tablet_bootstrap.cc:492] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:02.323433 21597 leader_election.cc:290] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:02.324604 21528 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:02.325055 21568 ts_tablet_manager.cc:1434] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.002s
I20260709 12:15:02.326346 21568 tablet_bootstrap.cc:492] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:02.329084 21275 catalog_manager.cc:5697] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:02.331880 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.332631 21378 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:02.332747 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.333436 21453 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:02.337385 21570 tablet_bootstrap.cc:654] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.330840 21417 leader_election.cc:304] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:02.339711 21604 raft_consensus.cc:2804] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.340065 21568 tablet_bootstrap.cc:654] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.340114 21604 raft_consensus.cc:493] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.341085 21604 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.341382 21379 raft_consensus.cc:2468] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:02.341755 21607 raft_consensus.cc:515] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.344597 21607 leader_election.cc:290] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.347126 21577 raft_consensus.cc:2749] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:02.348016 21494 leader_election.cc:304] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.349131 21577 raft_consensus.cc:2804] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.349509 21577 raft_consensus.cc:493] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.349488 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.350129 21577 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:02.351639 21417 leader_election.cc:343] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:02.360133 21604 raft_consensus.cc:515] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.363870 21604 leader_election.cc:290] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.365578 21577 raft_consensus.cc:515] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.366937 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.367918 21528 raft_consensus.cc:2393] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aa209a8923e14e68b1ba654178cedee3 in current term 1: Already voted for candidate a9ed2c75babe4e49a17ab052415d1434 in this term.
I20260709 12:15:02.368954 21570 tablet_bootstrap.cc:492] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:02.369580 21570 ts_tablet_manager.cc:1403] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.048s	user 0.007s	sys 0.017s
I20260709 12:15:02.370110 21569 tablet_bootstrap.cc:492] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.370698 21569 ts_tablet_manager.cc:1403] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.003s
I20260709 12:15:02.372165 21420 leader_election.cc:304] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.372859 21570 raft_consensus.cc:359] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.378201 21570 raft_consensus.cc:385] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.374043 21569 raft_consensus.cc:359] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.378551 21570 raft_consensus.cc:740] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.375372 21568 tablet_bootstrap.cc:492] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:02.375204 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.378862 21569 raft_consensus.cc:385] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.378939 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.379374 21379 raft_consensus.cc:3060] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.379495 21569 raft_consensus.cc:740] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.379673 21378 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.379391 21570 consensus_queue.cc:260] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.380664 21569 consensus_queue.cc:260] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.382144 21570 ts_tablet_manager.cc:1434] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:15:02.383214 21570 tablet_bootstrap.cc:492] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:02.386533 21604 raft_consensus.cc:2804] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.387212 21604 raft_consensus.cc:697] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:02.388167 21604 consensus_queue.cc:237] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.391695 21378 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:02.393018 21494 leader_election.cc:304] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.393819 21597 raft_consensus.cc:2804] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.394320 21597 raft_consensus.cc:697] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:02.394986 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.395725 21453 raft_consensus.cc:2393] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a9ed2c75babe4e49a17ab052415d1434 in current term 1: Already voted for candidate aa209a8923e14e68b1ba654178cedee3 in this term.
I20260709 12:15:02.395277 21597 consensus_queue.cc:237] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.388619 21379 raft_consensus.cc:2468] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:02.397715 21569 ts_tablet_manager.cc:1434] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:15:02.378651 21577 leader_election.cc:290] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:02.389515 21568 ts_tablet_manager.cc:1403] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.007s
I20260709 12:15:02.377481 21527 raft_consensus.cc:2468] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:02.399091 21569 tablet_bootstrap.cc:492] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:02.389866 21570 tablet_bootstrap.cc:654] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.406585 21420 leader_election.cc:304] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: a9ed2c75babe4e49a17ab052415d1434
I20260709 12:15:02.407835 21568 raft_consensus.cc:359] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.408819 21607 raft_consensus.cc:2804] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.409063 21568 raft_consensus.cc:385] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.409459 21607 raft_consensus.cc:697] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:02.409574 21568 raft_consensus.cc:740] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.410568 21607 consensus_queue.cc:237] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.410772 21568 consensus_queue.cc:260] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.416738 21569 tablet_bootstrap.cc:654] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.418797 21568 ts_tablet_manager.cc:1434] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.003s
I20260709 12:15:02.407697 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.421332 21379 raft_consensus.cc:2393] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aa209a8923e14e68b1ba654178cedee3 in current term 1: Already voted for candidate a9ed2c75babe4e49a17ab052415d1434 in this term.
I20260709 12:15:02.422926 21568 tablet_bootstrap.cc:492] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:02.425318 21420 leader_election.cc:304] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3; no voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.426326 21604 raft_consensus.cc:2749] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:02.430639 21275 catalog_manager.cc:5697] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.433000 21569 tablet_bootstrap.cc:492] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.433755 21569 ts_tablet_manager.cc:1403] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.002s
I20260709 12:15:02.435587 21276 catalog_manager.cc:5697] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:02.438150 21569 raft_consensus.cc:359] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.439427 21569 raft_consensus.cc:385] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.439924 21569 raft_consensus.cc:740] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.441443 21569 consensus_queue.cc:260] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.442340 21568 tablet_bootstrap.cc:654] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.442468 21577 raft_consensus.cc:493] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.443064 21577 raft_consensus.cc:515] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.447369 21569 ts_tablet_manager.cc:1434] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 12:15:02.448945 21569 tablet_bootstrap.cc:492] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:02.449254 21577 leader_election.cc:290] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:02.451248 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.455438 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.456393 21379 raft_consensus.cc:2393] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9ed2c75babe4e49a17ab052415d1434 in current term 1: Already voted for candidate aa209a8923e14e68b1ba654178cedee3 in this term.
I20260709 12:15:02.459333 21569 tablet_bootstrap.cc:654] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.459982 21494 leader_election.cc:304] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434; no voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.460448 21568 tablet_bootstrap.cc:492] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:02.461073 21568 ts_tablet_manager.cc:1403] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.001s
I20260709 12:15:02.461473 21577 raft_consensus.cc:3060] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.464794 21568 raft_consensus.cc:359] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.465893 21568 raft_consensus.cc:385] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.466269 21568 raft_consensus.cc:740] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.466440 21275 catalog_manager.cc:5697] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.469707 21582 raft_consensus.cc:493] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.470424 21582 raft_consensus.cc:515] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.471695 21577 raft_consensus.cc:2749] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:02.475410 21576 raft_consensus.cc:493] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.475965 21576 raft_consensus.cc:515] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.476060 21582 leader_election.cc:290] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.478083 21576 leader_election.cc:290] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.478394 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.478161 21568 consensus_queue.cc:260] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.479175 21527 raft_consensus.cc:2468] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:02.479912 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.480649 21528 raft_consensus.cc:2468] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:02.480909 21342 leader_election.cc:304] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.481855 21568 ts_tablet_manager.cc:1434] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 12:15:02.482343 21342 leader_election.cc:304] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.482729 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.483286 21582 raft_consensus.cc:2804] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.483386 21568 tablet_bootstrap.cc:492] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:02.483713 21582 raft_consensus.cc:493] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.483826 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.484205 21582 raft_consensus.cc:3060] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.484542 21453 raft_consensus.cc:2468] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:02.483453 21454 raft_consensus.cc:2468] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:02.492949 21568 tablet_bootstrap.cc:654] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.493696 21576 raft_consensus.cc:2804] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.494192 21576 raft_consensus.cc:493] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.493089 21582 raft_consensus.cc:515] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.494602 21576 raft_consensus.cc:3060] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.496068 21570 tablet_bootstrap.cc:492] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:02.496832 21570 ts_tablet_manager.cc:1403] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.114s	user 0.033s	sys 0.003s
I20260709 12:15:02.497594 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.498276 21454 raft_consensus.cc:3060] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.501219 21570 raft_consensus.cc:359] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.502218 21570 raft_consensus.cc:385] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.502568 21570 raft_consensus.cc:740] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.506480 21570 consensus_queue.cc:260] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.511487 21454 raft_consensus.cc:2468] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:02.512383 21576 raft_consensus.cc:515] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.513773 21582 leader_election.cc:290] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.514111 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.514967 21528 raft_consensus.cc:3060] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.516244 21576 leader_election.cc:290] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.519289 21568 tablet_bootstrap.cc:492] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:02.519120 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.519865 21527 raft_consensus.cc:3060] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.520709 21344 leader_election.cc:304] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.522253 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.522965 21454 raft_consensus.cc:3060] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.524713 21569 tablet_bootstrap.cc:492] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.525326 21569 ts_tablet_manager.cc:1403] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.077s	user 0.027s	sys 0.035s
I20260709 12:15:02.528128 21528 raft_consensus.cc:2468] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:02.528524 21570 ts_tablet_manager.cc:1434] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 12:15:02.529340 21582 raft_consensus.cc:2804] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.529848 21454 raft_consensus.cc:2468] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:02.529917 21582 raft_consensus.cc:697] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:02.531282 21344 leader_election.cc:304] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.531949 21569 raft_consensus.cc:359] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.532552 21568 ts_tablet_manager.cc:1403] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.000s
I20260709 12:15:02.532811 21569 raft_consensus.cc:385] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.533169 21569 raft_consensus.cc:740] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.533257 21576 raft_consensus.cc:2804] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.532172 21582 consensus_queue.cc:237] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.534152 21569 consensus_queue.cc:260] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.536207 21568 raft_consensus.cc:359] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.537046 21568 raft_consensus.cc:385] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.537382 21568 raft_consensus.cc:740] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.538262 21568 consensus_queue.cc:260] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.540262 21568 ts_tablet_manager.cc:1434] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:15:02.541141 21569 ts_tablet_manager.cc:1434] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:15:02.542361 21569 tablet_bootstrap.cc:492] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:02.528848 21527 raft_consensus.cc:2468] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:02.550043 21576 raft_consensus.cc:697] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:02.548971 21569 tablet_bootstrap.cc:654] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:02.551138 21576 consensus_queue.cc:237] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.562786 21569 tablet_bootstrap.cc:492] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:02.563459 21569 ts_tablet_manager.cc:1403] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:15:02.566030 21569 raft_consensus.cc:359] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.566533 21569 raft_consensus.cc:385] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:02.566691 21569 raft_consensus.cc:740] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:02.567301 21569 consensus_queue.cc:260] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.569157 21569 ts_tablet_manager.cc:1434] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:15:02.568838 21276 catalog_manager.cc:5697] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.571120 21276 catalog_manager.cc:5697] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.575812 21577 raft_consensus.cc:493] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.576553 21577 raft_consensus.cc:515] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.577561 21604 raft_consensus.cc:493] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.578166 21604 raft_consensus.cc:515] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.579663 21577 leader_election.cc:290] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483)
I20260709 12:15:02.580600 21604 leader_election.cc:290] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.583597 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.584043 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:02.584333 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d90860574723485e8d764982f9abbf13" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:02.584789 21454 raft_consensus.cc:2468] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:02.584403 21379 raft_consensus.cc:2468] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:02.586158 21493 leader_election.cc:304] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:02.586438 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d90860574723485e8d764982f9abbf13" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:02.587159 21378 raft_consensus.cc:2468] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:02.585290 21527 raft_consensus.cc:2468] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:02.588104 21420 leader_election.cc:304] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.587579 21577 raft_consensus.cc:2804] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.589006 21604 raft_consensus.cc:2804] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:02.589648 21577 raft_consensus.cc:493] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.589746 21604 raft_consensus.cc:493] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.590040 21577 raft_consensus.cc:3060] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.590255 21604 raft_consensus.cc:3060] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.596741 21604 raft_consensus.cc:515] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.596740 21577 raft_consensus.cc:515] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.598796 21604 leader_election.cc:290] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.599821 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d90860574723485e8d764982f9abbf13" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.600159 21577 leader_election.cc:290] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483)
I20260709 12:15:02.599821 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.600016 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d90860574723485e8d764982f9abbf13" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.600783 21454 raft_consensus.cc:3060] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.601426 21527 raft_consensus.cc:3060] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.600912 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.600600 21378 raft_consensus.cc:3060] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.602167 21379 raft_consensus.cc:3060] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.609838 21378 raft_consensus.cc:2468] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:02.609838 21527 raft_consensus.cc:2468] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:02.609966 21454 raft_consensus.cc:2468] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:02.610503 21379 raft_consensus.cc:2468] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:02.611856 21417 leader_election.cc:304] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:02.612135 21493 leader_election.cc:304] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:02.613054 21577 raft_consensus.cc:2804] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.613062 21604 raft_consensus.cc:2804] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:02.613657 21577 raft_consensus.cc:697] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:02.614279 21604 raft_consensus.cc:697] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:02.615152 21577 consensus_queue.cc:237] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:02.615305 21604 consensus_queue.cc:237] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.625674 21275 catalog_manager.cc:5697] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.625730 21276 catalog_manager.cc:5697] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.676079 21527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
mode: GRACEFUL
new_leader_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:42938
I20260709 12:15:02.676772 21527 raft_consensus.cc:606] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Received request to transfer leadership to TS ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.683070 21527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
mode: GRACEFUL
new_leader_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:42938
I20260709 12:15:02.683667 21527 raft_consensus.cc:606] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Received request to transfer leadership to TS ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.702610 21454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
mode: GRACEFUL
new_leader_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:58696
I20260709 12:15:02.703349 21454 raft_consensus.cc:606] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Received request to transfer leadership to TS ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:02.703606 21576 consensus_queue.cc:1048] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:02.719875 21454 raft_consensus.cc:3060] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:02.724663 21379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:54854
I20260709 12:15:02.725642 21379 raft_consensus.cc:606] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Received request to transfer leadership to TS aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:02.731483 21576 consensus_queue.cc:1048] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:02.733448 21577 consensus_queue.cc:1048] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:02.740690 21527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
mode: GRACEFUL
new_leader_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:42938
I20260709 12:15:02.741369 21527 raft_consensus.cc:606] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Received request to transfer leadership to TS aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:02.750813 21577 consensus_queue.cc:1048] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:02.754594 21602 raft_consensus.cc:993] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: : Instructing follower aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:02.755195 21597 raft_consensus.cc:1081] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Signalling peer aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:02.756438 21378 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:54854
I20260709 12:15:02.757200 21378 raft_consensus.cc:606] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Received request to transfer leadership to TS a9ed2c75babe4e49a17ab052415d1434
I20260709 12:15:02.759492 21453 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:58686
I20260709 12:15:02.761505 21453 raft_consensus.cc:493] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:02.762015 21453 raft_consensus.cc:3060] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.771469 21453 raft_consensus.cc:515] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.774004 21453 leader_election.cc:290] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.776736 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:02.777412 21379 raft_consensus.cc:3060] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.780197 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "002b11df78754235ae83eb63aaebe2c9" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.781333 21527 raft_consensus.cc:3055] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:02.781699 21527 raft_consensus.cc:740] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:02.783952 21527 consensus_queue.cc:260] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.785436 21527 raft_consensus.cc:3060] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.786804 21379 raft_consensus.cc:2468] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:02.788094 21420 leader_election.cc:304] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.789057 21604 raft_consensus.cc:2804] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:02.789597 21604 raft_consensus.cc:697] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:02.790498 21604 consensus_queue.cc:237] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.795318 21527 raft_consensus.cc:2468] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:02.795624 21577 consensus_queue.cc:1048] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:02.798148 21604 consensus_queue.cc:1048] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:02.801600 21276 catalog_manager.cc:5697] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 1 to 2, leader changed from a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67) to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.813863 21577 consensus_queue.cc:1048] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:02.823304 21574 consensus_queue.cc:1048] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:02.835328 21577 raft_consensus.cc:993] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: : Instructing follower ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:02.835911 21577 raft_consensus.cc:1081] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Signalling peer ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:02.837265 21574 raft_consensus.cc:993] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: : Instructing follower ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:02.837859 21574 raft_consensus.cc:1081] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Signalling peer ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:02.838294 21378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:54846
I20260709 12:15:02.839047 21378 raft_consensus.cc:493] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:02.839512 21378 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.840224 21379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:54848
I20260709 12:15:02.840941 21379 raft_consensus.cc:493] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:02.841348 21379 raft_consensus.cc:3060] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.847350 21378 raft_consensus.cc:515] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.849133 21379 raft_consensus.cc:515] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.849933 21378 leader_election.cc:290] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.850374 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.850699 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.851117 21453 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.851480 21527 raft_consensus.cc:3055] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:02.851886 21527 raft_consensus.cc:740] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:02.852571 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:02.852936 21379 leader_election.cc:290] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:02.853291 21452 raft_consensus.cc:3055] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:02.853003 21527 consensus_queue.cc:260] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.853706 21452 raft_consensus.cc:740] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:02.854533 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca48b558d1c438c868aa42cdff43269" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:02.854816 21527 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.855193 21528 raft_consensus.cc:3060] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.854874 21452 consensus_queue.cc:260] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.856346 21452 raft_consensus.cc:3060] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:02.859877 21453 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:02.861279 21344 leader_election.cc:304] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.862306 21582 raft_consensus.cc:2804] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:02.862835 21582 raft_consensus.cc:697] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:02.863448 21527 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:02.864063 21528 raft_consensus.cc:2468] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:02.863771 21582 consensus_queue.cc:237] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.865640 21342 leader_election.cc:304] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:02.865841 21452 raft_consensus.cc:2468] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:02.867106 21622 raft_consensus.cc:2804] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:02.867857 21622 raft_consensus.cc:697] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:02.868844 21622 consensus_queue.cc:237] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:02.874894 21274 catalog_manager.cc:5697] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 1 to 2, leader changed from a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67) to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.889508 21274 catalog_manager.cc:5697] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 1 to 2, leader changed from aa209a8923e14e68b1ba654178cedee3 (127.9.11.66) to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:02.968914 21604 consensus_queue.cc:1048] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:02.982293 21610 consensus_queue.cc:1048] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:02.995786 21622 consensus_queue.cc:1048] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.009413 21622 consensus_queue.cc:1048] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.022006 21597 consensus_queue.cc:1048] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:03.025274 21576 raft_consensus.cc:993] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: : Instructing follower a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:03.025899 21622 raft_consensus.cc:1081] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Signalling peer a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:03.028427 21527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:42910
I20260709 12:15:03.029085 21527 raft_consensus.cc:493] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:03.029515 21527 raft_consensus.cc:3060] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.040758 21577 consensus_queue.cc:1048] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.043169 21527 raft_consensus.cc:515] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:03.044062 21610 consensus_queue.cc:1048] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.051690 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:03.052130 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d74552227943f4afdee2a1ea3c547d" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:03.052520 21453 raft_consensus.cc:3060] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.053012 21377 raft_consensus.cc:3055] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:03.053376 21377 raft_consensus.cc:740] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:03.054342 21377 consensus_queue.cc:260] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:03.055145 21527 leader_election.cc:290] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:03.055692 21377 raft_consensus.cc:3060] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.070142 21610 consensus_queue.cc:1048] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.070611 21453 raft_consensus.cc:2468] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 2.
I20260709 12:15:03.072266 21377 raft_consensus.cc:2468] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 2.
I20260709 12:15:03.073693 21494 leader_election.cc:304] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:03.074698 21577 raft_consensus.cc:2804] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:03.075307 21577 raft_consensus.cc:697] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 2 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:03.076390 21577 consensus_queue.cc:237] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:03.083087 21602 raft_consensus.cc:993] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: : Instructing follower ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:03.083642 21577 raft_consensus.cc:1081] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Signalling peer ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:03.085872 21379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:54846
I20260709 12:15:03.086438 21379 raft_consensus.cc:493] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:03.086824 21379 raft_consensus.cc:3060] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.087872 21274 catalog_manager.cc:5697] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 1 to 2, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:03.093389 21379 raft_consensus.cc:515] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:03.095800 21379 leader_election.cc:290] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:03.097133 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:03.097988 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de0f5c45176b4a82a53549389e30b72e" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:03.098698 21527 raft_consensus.cc:3055] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:03.097965 21453 raft_consensus.cc:3060] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.099182 21527 raft_consensus.cc:740] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:03.100172 21527 consensus_queue.cc:260] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:03.101522 21527 raft_consensus.cc:3060] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.106576 21453 raft_consensus.cc:2468] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:03.107892 21344 leader_election.cc:304] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:03.108094 21527 raft_consensus.cc:2468] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:03.108902 21622 raft_consensus.cc:2804] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:03.109483 21622 raft_consensus.cc:697] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:03.110311 21622 consensus_queue.cc:237] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } }
I20260709 12:15:03.117453 21276 catalog_manager.cc:5697] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 1 to 2, leader changed from a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67) to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:03.138827 21622 consensus_queue.cc:1048] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.147347 21576 raft_consensus.cc:993] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: : Instructing follower aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:03.147940 21623 raft_consensus.cc:1081] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Signalling peer aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:03.150547 21452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:58684
I20260709 12:15:03.154062 21452 raft_consensus.cc:493] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:03.153787 21623 consensus_queue.cc:1048] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:03.154578 21452 raft_consensus.cc:3060] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.163851 21452 raft_consensus.cc:515] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:03.167378 21452 leader_election.cc:290] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:03.168067 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:03.168653 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a0c0185f6e4e9193531352768d711a" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:03.168910 21527 raft_consensus.cc:3060] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.169400 21379 raft_consensus.cc:3055] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:03.169747 21379 raft_consensus.cc:740] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:03.170679 21379 consensus_queue.cc:260] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:03.172260 21379 raft_consensus.cc:3060] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:03.181710 21379 raft_consensus.cc:2468] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:03.181919 21527 raft_consensus.cc:2468] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:03.183123 21420 leader_election.cc:304] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:03.184198 21610 raft_consensus.cc:2804] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:03.184732 21610 raft_consensus.cc:697] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:03.185595 21610 consensus_queue.cc:237] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:03.193135 21276 catalog_manager.cc:5697] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 1 to 2, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:03.219875 21379 raft_consensus.cc:1275] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.221560 21610 consensus_queue.cc:1048] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.231231 21527 raft_consensus.cc:1275] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.233229 21604 consensus_queue.cc:1048] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.277606 21452 raft_consensus.cc:1275] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.279294 21622 consensus_queue.cc:1048] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.288434 21527 raft_consensus.cc:1275] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.290248 21622 consensus_queue.cc:1048] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.410517 21527 raft_consensus.cc:1275] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.412218 21622 consensus_queue.cc:1048] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.422462 21452 raft_consensus.cc:1275] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.424830 21623 consensus_queue.cc:1048] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.595474 21379 raft_consensus.cc:1275] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.597160 21602 consensus_queue.cc:1048] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.608808 21452 raft_consensus.cc:1275] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.611538 21655 consensus_queue.cc:1048] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:03.684903 21452 raft_consensus.cc:1275] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.686427 21623 consensus_queue.cc:1048] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.695719 21527 raft_consensus.cc:1275] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.697381 21582 consensus_queue.cc:1048] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.729511 21527 raft_consensus.cc:1275] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.731180 21610 consensus_queue.cc:1048] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:03.739960 21379 raft_consensus.cc:1275] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:03.742439 21604 consensus_queue.cc:1048] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:05.797711 21276 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59308:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:05.801316 21276 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:05.873221 21361 tablet_service.cc:1511] Processing CreateTablet for tablet ca7dfc01c7344a858d588558dcc56486 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:05.875718 21361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca7dfc01c7344a858d588558dcc56486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.878540 21442 tablet_service.cc:1511] Processing CreateTablet for tablet ca7dfc01c7344a858d588558dcc56486 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:05.880664 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca7dfc01c7344a858d588558dcc56486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.883119 21444 tablet_service.cc:1511] Processing CreateTablet for tablet 11ffa400581a495dbb4717536e97c3b4 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:05.884981 21444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ffa400581a495dbb4717536e97c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.887176 21518 tablet_service.cc:1511] Processing CreateTablet for tablet ca7dfc01c7344a858d588558dcc56486 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:05.889318 21518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca7dfc01c7344a858d588558dcc56486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.890688 21440 tablet_service.cc:1511] Processing CreateTablet for tablet 96d3292d44834c4490fef866d135bc27 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:05.893793 21513 tablet_service.cc:1511] Processing CreateTablet for tablet 11ffa400581a495dbb4717536e97c3b4 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:05.899871 21436 tablet_service.cc:1511] Processing CreateTablet for tablet 4a76cabcbbe0432eab77588d04b43a4a (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:05.901612 21436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a76cabcbbe0432eab77588d04b43a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.895769 21513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ffa400581a495dbb4717536e97c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.908515 21510 tablet_service.cc:1511] Processing CreateTablet for tablet 4a76cabcbbe0432eab77588d04b43a4a (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:05.906848 21511 tablet_service.cc:1511] Processing CreateTablet for tablet 96d3292d44834c4490fef866d135bc27 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:05.927444 21441 tablet_service.cc:1511] Processing CreateTablet for tablet 8a907452714e4cfdaa3631724eea1969 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:05.928766 21512 tablet_service.cc:1511] Processing CreateTablet for tablet 8a907452714e4cfdaa3631724eea1969 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:05.930114 21440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96d3292d44834c4490fef866d135bc27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.931022 21514 tablet_service.cc:1511] Processing CreateTablet for tablet 78b1c4f1a6724242be3bad15722432c2 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:05.931648 21516 tablet_service.cc:1511] Processing CreateTablet for tablet 697bb0a36e144873afd023990ca9660a (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:05.935387 21439 tablet_service.cc:1511] Processing CreateTablet for tablet 78b1c4f1a6724242be3bad15722432c2 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:05.939011 21438 tablet_service.cc:1511] Processing CreateTablet for tablet 697bb0a36e144873afd023990ca9660a (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:05.942081 21443 tablet_service.cc:1511] Processing CreateTablet for tablet bf33fa7474b540eda2a042f357a5b947 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:05.943602 21514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b1c4f1a6724242be3bad15722432c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.945971 21516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 697bb0a36e144873afd023990ca9660a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.948493 21512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a907452714e4cfdaa3631724eea1969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.949263 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf33fa7474b540eda2a042f357a5b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.952005 21511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96d3292d44834c4490fef866d135bc27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.952399 21439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b1c4f1a6724242be3bad15722432c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.954805 21438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 697bb0a36e144873afd023990ca9660a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.955796 21437 tablet_service.cc:1511] Processing CreateTablet for tablet 6266bcfcf22444619a61bffac71edb32 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:05.957607 21437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6266bcfcf22444619a61bffac71edb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.963436 21510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a76cabcbbe0432eab77588d04b43a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.967240 21441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a907452714e4cfdaa3631724eea1969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.977150 21517 tablet_service.cc:1511] Processing CreateTablet for tablet bf33fa7474b540eda2a042f357a5b947 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:05.977963 21515 tablet_service.cc:1511] Processing CreateTablet for tablet 6266bcfcf22444619a61bffac71edb32 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:05.979105 21517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf33fa7474b540eda2a042f357a5b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:05.980259 21515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6266bcfcf22444619a61bffac71edb32. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:05.999331 21342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:35351, user_credentials={real_user=slave}} blocked reactor thread for 123580us
I20260709 12:15:06.004043 21369 tablet_service.cc:1511] Processing CreateTablet for tablet 11ffa400581a495dbb4717536e97c3b4 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:06.006353 21369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ffa400581a495dbb4717536e97c3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:06.011278 21366 tablet_service.cc:1511] Processing CreateTablet for tablet 96d3292d44834c4490fef866d135bc27 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:06.013490 21366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96d3292d44834c4490fef866d135bc27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:06.025786 21365 tablet_service.cc:1511] Processing CreateTablet for tablet 4a76cabcbbe0432eab77588d04b43a4a (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:06.027794 21365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a76cabcbbe0432eab77588d04b43a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:06.049652 21368 tablet_service.cc:1511] Processing CreateTablet for tablet 697bb0a36e144873afd023990ca9660a (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:06.051751 21362 tablet_service.cc:1511] Processing CreateTablet for tablet bf33fa7474b540eda2a042f357a5b947 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:06.059571 21363 tablet_service.cc:1511] Processing CreateTablet for tablet 8a907452714e4cfdaa3631724eea1969 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:06.062059 21364 tablet_service.cc:1511] Processing CreateTablet for tablet 78b1c4f1a6724242be3bad15722432c2 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:06.064332 21367 tablet_service.cc:1511] Processing CreateTablet for tablet 6266bcfcf22444619a61bffac71edb32 (DEFAULT_TABLE table=soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:06.068563 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 697bb0a36e144873afd023990ca9660a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:06.073329 21363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a907452714e4cfdaa3631724eea1969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:06.075803 21367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6266bcfcf22444619a61bffac71edb32. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:06.076623 21325 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:06.078162 21364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b1c4f1a6724242be3bad15722432c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:06.079588 21362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf33fa7474b540eda2a042f357a5b947. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:06.090409 21420 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.65:40483, user_credentials={real_user=slave}} blocked reactor thread for 127118us
I20260709 12:15:06.193099 21675 tablet_bootstrap.cc:492] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.210500 21675 tablet_bootstrap.cc:654] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.216429 21676 tablet_bootstrap.cc:492] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.220049 21677 tablet_bootstrap.cc:492] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.227617 21675 tablet_bootstrap.cc:492] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.228219 21677 tablet_bootstrap.cc:654] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.228374 21675 ts_tablet_manager.cc:1403] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.004s
I20260709 12:15:06.232112 21675 raft_consensus.cc:359] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.233338 21675 raft_consensus.cc:385] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.233697 21675 raft_consensus.cc:740] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.234697 21675 consensus_queue.cc:260] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.235605 21676 tablet_bootstrap.cc:654] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.237288 21677 tablet_bootstrap.cc:492] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.237921 21675 ts_tablet_manager.cc:1434] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:15:06.238034 21677 ts_tablet_manager.cc:1403] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.005s
I20260709 12:15:06.241860 21677 raft_consensus.cc:359] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.242689 21677 raft_consensus.cc:385] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.243084 21677 raft_consensus.cc:740] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.243876 21677 consensus_queue.cc:260] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.245042 21675 tablet_bootstrap.cc:492] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.249037 21677 ts_tablet_manager.cc:1434] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 12:15:06.250483 21677 tablet_bootstrap.cc:492] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.254882 21675 tablet_bootstrap.cc:654] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.244604 21676 tablet_bootstrap.cc:492] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.259817 21677 tablet_bootstrap.cc:654] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.260077 21676 ts_tablet_manager.cc:1403] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.000s
I20260709 12:15:06.264773 21676 raft_consensus.cc:359] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.265933 21676 raft_consensus.cc:385] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.266463 21676 raft_consensus.cc:740] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.267738 21676 consensus_queue.cc:260] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.268752 21675 tablet_bootstrap.cc:492] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.269598 21675 ts_tablet_manager.cc:1403] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.006s
I20260709 12:15:06.273334 21675 raft_consensus.cc:359] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.274178 21675 raft_consensus.cc:385] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.274497 21675 raft_consensus.cc:740] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.275589 21675 consensus_queue.cc:260] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.278232 21675 ts_tablet_manager.cc:1434] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:15:06.279778 21675 tablet_bootstrap.cc:492] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.292349 21675 tablet_bootstrap.cc:654] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.295408 21676 ts_tablet_manager.cc:1434] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.005s
I20260709 12:15:06.304188 21676 tablet_bootstrap.cc:492] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.308843 21677 tablet_bootstrap.cc:492] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.309959 21677 ts_tablet_manager.cc:1403] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.060s	user 0.026s	sys 0.029s
I20260709 12:15:06.312935 21676 tablet_bootstrap.cc:654] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.315101 21677 raft_consensus.cc:359] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.316373 21677 raft_consensus.cc:385] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.316834 21677 raft_consensus.cc:740] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.317742 21677 consensus_queue.cc:260] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.320526 21677 ts_tablet_manager.cc:1434] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:15:06.321934 21677 tablet_bootstrap.cc:492] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.330142 21676 tablet_bootstrap.cc:492] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.330497 21677 tablet_bootstrap.cc:654] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.330883 21676 ts_tablet_manager.cc:1403] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.007s
I20260709 12:15:06.330161 21675 tablet_bootstrap.cc:492] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.331686 21675 ts_tablet_manager.cc:1403] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.052s	user 0.020s	sys 0.012s
I20260709 12:15:06.334811 21676 raft_consensus.cc:359] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.335131 21675 raft_consensus.cc:359] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.335719 21676 raft_consensus.cc:385] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.335846 21675 raft_consensus.cc:385] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.336074 21676 raft_consensus.cc:740] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.336166 21675 raft_consensus.cc:740] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.337087 21675 consensus_queue.cc:260] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.337160 21676 consensus_queue.cc:260] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.342336 21671 raft_consensus.cc:493] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.342840 21676 ts_tablet_manager.cc:1434] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:15:06.343364 21671 raft_consensus.cc:515] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.344205 21676 tablet_bootstrap.cc:492] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.348487 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
W20260709 12:15:06.351735 21344 leader_election.cc:343] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.358271 21676 tablet_bootstrap.cc:654] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.352018 21675 ts_tablet_manager.cc:1434] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:15:06.360285 21675 tablet_bootstrap.cc:492] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.371320 21675 tablet_bootstrap.cc:654] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.381244 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:06.382419 21528 raft_consensus.cc:2468] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.384644 21342 leader_election.cc:304] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:06.385417 21675 tablet_bootstrap.cc:492] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.386224 21675 ts_tablet_manager.cc:1403] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.009s
I20260709 12:15:06.386317 21672 raft_consensus.cc:2804] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.387398 21672 raft_consensus.cc:493] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.387835 21672 raft_consensus.cc:3060] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.390938 21675 raft_consensus.cc:359] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.392109 21675 raft_consensus.cc:385] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.392505 21675 raft_consensus.cc:740] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.393786 21675 consensus_queue.cc:260] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.397222 21675 ts_tablet_manager.cc:1434] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.003s
I20260709 12:15:06.398738 21675 tablet_bootstrap.cc:492] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.409178 21675 tablet_bootstrap.cc:654] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.413324 21671 leader_election.cc:290] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.423593 21675 tablet_bootstrap.cc:492] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.424540 21675 ts_tablet_manager.cc:1403] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.007s
I20260709 12:15:06.426889 21655 raft_consensus.cc:493] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.427595 21655 raft_consensus.cc:515] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.429355 21675 raft_consensus.cc:359] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.430215 21655 leader_election.cc:290] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.430523 21675 raft_consensus.cc:385] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.431013 21675 raft_consensus.cc:740] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.431917 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.432114 21675 consensus_queue.cc:260] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.435810 21675 ts_tablet_manager.cc:1434] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.003s
I20260709 12:15:06.437335 21675 tablet_bootstrap.cc:492] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
W20260709 12:15:06.437426 21493 leader_election.cc:343] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.440924 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:06.442000 21379 raft_consensus.cc:2468] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:06.444247 21494 leader_election.cc:304] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:06.445418 21655 raft_consensus.cc:2804] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.445803 21655 raft_consensus.cc:493] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.446157 21655 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.454393 21675 tablet_bootstrap.cc:654] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.460225 21672 raft_consensus.cc:515] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.460631 21655 raft_consensus.cc:515] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.463613 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.467288 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
W20260709 12:15:06.468185 21344 leader_election.cc:343] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.467324 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
W20260709 12:15:06.471331 21493 leader_election.cc:343] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.468202 21379 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.473753 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:06.474373 21527 raft_consensus.cc:3060] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.475304 21672 leader_election.cc:290] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.481484 21675 tablet_bootstrap.cc:492] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.482316 21675 ts_tablet_manager.cc:1403] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.008s
I20260709 12:15:06.487346 21675 raft_consensus.cc:359] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.488422 21675 raft_consensus.cc:385] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.488826 21675 raft_consensus.cc:740] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.489982 21675 consensus_queue.cc:260] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.463706 21655 leader_election.cc:290] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.493029 21527 raft_consensus.cc:2468] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.493433 21655 raft_consensus.cc:493] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.494037 21655 raft_consensus.cc:515] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.494704 21342 leader_election.cc:304] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:06.495136 21379 raft_consensus.cc:2468] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:06.497350 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:06.497506 21494 leader_election.cc:304] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:06.497952 21676 tablet_bootstrap.cc:492] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.498556 21676 ts_tablet_manager.cc:1403] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.155s	user 0.049s	sys 0.080s
I20260709 12:15:06.498723 21672 raft_consensus.cc:2804] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:15:06.499485 21494 leader_election.cc:343] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.500197 21672 raft_consensus.cc:697] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:06.502270 21672 consensus_queue.cc:237] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.502785 21655 leader_election.cc:290] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.508014 21655 raft_consensus.cc:2804] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.506857 21677 tablet_bootstrap.cc:492] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.508618 21655 raft_consensus.cc:697] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:06.509625 21655 consensus_queue.cc:237] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.512601 21676 raft_consensus.cc:359] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.513367 21676 raft_consensus.cc:385] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.513830 21676 raft_consensus.cc:740] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.514659 21676 consensus_queue.cc:260] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.524319 21676 ts_tablet_manager.cc:1434] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
I20260709 12:15:06.526173 21676 tablet_bootstrap.cc:492] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.535106 21677 ts_tablet_manager.cc:1403] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.214s	user 0.052s	sys 0.059s
I20260709 12:15:06.536823 21676 tablet_bootstrap.cc:654] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.544160 21690 raft_consensus.cc:493] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.550935 21675 ts_tablet_manager.cc:1434] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.068s	user 0.032s	sys 0.031s
I20260709 12:15:06.552887 21675 tablet_bootstrap.cc:492] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.555262 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.553351 21690 raft_consensus.cc:515] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.560954 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ffa400581a495dbb4717536e97c3b4" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
W20260709 12:15:06.562310 21493 leader_election.cc:343] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.563040 21493 leader_election.cc:304] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434; no voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12
W20260709 12:15:06.563411 21494 leader_election.cc:343] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.564112 21655 raft_consensus.cc:2749] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:06.565590 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ffa400581a495dbb4717536e97c3b4" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.561228 21690 leader_election.cc:290] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.567268 21677 raft_consensus.cc:359] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.568132 21677 raft_consensus.cc:385] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:15:06.568857 21493 leader_election.cc:343] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.569448 21493 leader_election.cc:304] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434; no voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:06.568249 21276 catalog_manager.cc:5697] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:06.570287 21690 raft_consensus.cc:2749] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:06.570603 21677 raft_consensus.cc:740] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.571557 21675 tablet_bootstrap.cc:654] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.571641 21677 consensus_queue.cc:260] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.574838 21677 ts_tablet_manager.cc:1434] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:15:06.576185 21677 tablet_bootstrap.cc:492] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.576527 21274 catalog_manager.cc:5697] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:06.584849 21675 tablet_bootstrap.cc:492] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.584856 21677 tablet_bootstrap.cc:654] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.585673 21675 ts_tablet_manager.cc:1403] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.010s
I20260709 12:15:06.587419 21676 tablet_bootstrap.cc:492] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.589830 21675 raft_consensus.cc:359] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.590902 21675 raft_consensus.cc:385] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.591440 21675 raft_consensus.cc:740] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.592689 21675 consensus_queue.cc:260] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.595485 21676 ts_tablet_manager.cc:1403] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.070s	user 0.025s	sys 0.000s
I20260709 12:15:06.596172 21675 ts_tablet_manager.cc:1434] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:15:06.597378 21675 tablet_bootstrap.cc:492] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.597450 21677 tablet_bootstrap.cc:492] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.598053 21677 ts_tablet_manager.cc:1403] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 12:15:06.601612 21677 raft_consensus.cc:359] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.602473 21677 raft_consensus.cc:385] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.601687 21676 raft_consensus.cc:359] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.602806 21677 raft_consensus.cc:740] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.603289 21676 raft_consensus.cc:385] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.603876 21676 raft_consensus.cc:740] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.603938 21677 consensus_queue.cc:260] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.605827 21675 tablet_bootstrap.cc:654] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.605329 21676 consensus_queue.cc:260] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.607245 21677 ts_tablet_manager.cc:1434] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 12:15:06.609560 21676 ts_tablet_manager.cc:1434] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:15:06.611081 21676 tablet_bootstrap.cc:492] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.614185 21677 tablet_bootstrap.cc:492] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.620078 21676 tablet_bootstrap.cc:654] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.624890 21677 tablet_bootstrap.cc:654] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.638701 21677 tablet_bootstrap.cc:492] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.639364 21677 ts_tablet_manager.cc:1403] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.007s
I20260709 12:15:06.639650 21675 tablet_bootstrap.cc:492] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.640326 21675 ts_tablet_manager.cc:1403] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.011s
I20260709 12:15:06.642863 21677 raft_consensus.cc:359] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.643875 21677 raft_consensus.cc:385] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.644163 21677 raft_consensus.cc:740] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.644318 21675 raft_consensus.cc:359] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.645105 21675 raft_consensus.cc:385] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.645423 21675 raft_consensus.cc:740] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.645951 21677 consensus_queue.cc:260] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.646399 21675 consensus_queue.cc:260] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.648556 21676 tablet_bootstrap.cc:492] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.649080 21675 ts_tablet_manager.cc:1434] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:15:06.649147 21677 ts_tablet_manager.cc:1434] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:15:06.649214 21676 ts_tablet_manager.cc:1403] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.013s
I20260709 12:15:06.650516 21675 tablet_bootstrap.cc:492] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: Bootstrap starting.
I20260709 12:15:06.650517 21677 tablet_bootstrap.cc:492] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.650868 21690 raft_consensus.cc:493] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.651458 21690 raft_consensus.cc:515] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.655231 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.655982 21452 raft_consensus.cc:2468] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:06.658870 21677 tablet_bootstrap.cc:654] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.662267 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:06.658870 21675 tablet_bootstrap.cc:654] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.662848 21690 leader_election.cc:290] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.663676 21493 leader_election.cc:304] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
W20260709 12:15:06.664636 21494 leader_election.cc:343] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.669240 21676 raft_consensus.cc:359] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.670287 21676 raft_consensus.cc:385] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.670642 21676 raft_consensus.cc:740] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.671902 21676 consensus_queue.cc:260] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.669265 21690 raft_consensus.cc:2804] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.673432 21690 raft_consensus.cc:493] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.673861 21690 raft_consensus.cc:3060] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.678540 21675 tablet_bootstrap.cc:492] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: No bootstrap required, opened a new log
I20260709 12:15:06.679139 21675 ts_tablet_manager.cc:1403] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.006s
I20260709 12:15:06.681946 21676 ts_tablet_manager.cc:1434] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:15:06.683512 21676 tablet_bootstrap.cc:492] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.684520 21675 raft_consensus.cc:359] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.685298 21675 raft_consensus.cc:385] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.685573 21675 raft_consensus.cc:740] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.686395 21675 consensus_queue.cc:260] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.688702 21675 ts_tablet_manager.cc:1434] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:15:06.691133 21676 tablet_bootstrap.cc:654] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.691422 21690 raft_consensus.cc:515] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.693822 21690 leader_election.cc:290] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.693919 21677 tablet_bootstrap.cc:492] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.694592 21677 ts_tablet_manager.cc:1403] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.009s
I20260709 12:15:06.695986 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.696638 21452 raft_consensus.cc:3060] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.697140 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
W20260709 12:15:06.699113 21494 leader_election.cc:343] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:06.700384 21677 raft_consensus.cc:359] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.701403 21677 raft_consensus.cc:385] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.701854 21677 raft_consensus.cc:740] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.702656 21677 consensus_queue.cc:260] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.707191 21452 raft_consensus.cc:2468] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:06.708745 21493 leader_election.cc:304] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:06.709514 21676 tablet_bootstrap.cc:492] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.710083 21676 ts_tablet_manager.cc:1403] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.005s
I20260709 12:15:06.710050 21690 raft_consensus.cc:2804] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.711593 21690 raft_consensus.cc:697] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:06.711897 21677 ts_tablet_manager.cc:1434] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 12:15:06.713128 21677 tablet_bootstrap.cc:492] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.712704 21690 consensus_queue.cc:237] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.714134 21676 raft_consensus.cc:359] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.714926 21676 raft_consensus.cc:385] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.715252 21676 raft_consensus.cc:740] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.716132 21676 consensus_queue.cc:260] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.719676 21676 ts_tablet_manager.cc:1434] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:15:06.720224 21655 raft_consensus.cc:493] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.720827 21676 tablet_bootstrap.cc:492] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.721906 21677 tablet_bootstrap.cc:654] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.720801 21655 raft_consensus.cc:515] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.726217 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:06.727016 21379 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:06.728566 21494 leader_election.cc:304] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.729880 21691 raft_consensus.cc:2804] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.730312 21691 raft_consensus.cc:493] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.730693 21691 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.730873 21676 tablet_bootstrap.cc:654] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.736696 21677 tablet_bootstrap.cc:492] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.737257 21677 ts_tablet_manager.cc:1403] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.003s
I20260709 12:15:06.737659 21655 leader_election.cc:290] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.739070 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.739820 21453 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:06.740602 21677 raft_consensus.cc:359] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.741402 21677 raft_consensus.cc:385] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.741693 21677 raft_consensus.cc:740] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.742666 21677 consensus_queue.cc:260] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.748039 21677 ts_tablet_manager.cc:1434] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:15:06.755944 21677 tablet_bootstrap.cc:492] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.754763 21691 raft_consensus.cc:515] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.765583 21691 leader_election.cc:290] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.767040 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.768092 21453 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.768527 21677 tablet_bootstrap.cc:654] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.764667 21274 catalog_manager.cc:5697] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:06.775246 21676 tablet_bootstrap.cc:492] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.775874 21676 ts_tablet_manager.cc:1403] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.025s
I20260709 12:15:06.778394 21453 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:06.779819 21493 leader_election.cc:304] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:06.779789 21676 raft_consensus.cc:359] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.780797 21676 raft_consensus.cc:385] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.781054 21691 raft_consensus.cc:2804] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.781158 21676 raft_consensus.cc:740] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.781706 21691 raft_consensus.cc:697] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:06.782948 21691 consensus_queue.cc:237] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.782253 21676 consensus_queue.cc:260] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.787060 21677 tablet_bootstrap.cc:492] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.787289 21676 ts_tablet_manager.cc:1434] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:15:06.787607 21677 ts_tablet_manager.cc:1403] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.000s
I20260709 12:15:06.789987 21676 tablet_bootstrap.cc:492] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.789932 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:06.792070 21379 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.791024 21677 raft_consensus.cc:359] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.799157 21677 raft_consensus.cc:385] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.799497 21677 raft_consensus.cc:740] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.799778 21676 tablet_bootstrap.cc:654] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.800323 21677 consensus_queue.cc:260] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.800045 21275 catalog_manager.cc:5697] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:06.802129 21379 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:06.802742 21677 ts_tablet_manager.cc:1434] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:15:06.805191 21677 tablet_bootstrap.cc:492] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: Bootstrap starting.
I20260709 12:15:06.811976 21673 raft_consensus.cc:493] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.812427 21677 tablet_bootstrap.cc:654] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.812731 21673 raft_consensus.cc:515] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.819185 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d3292d44834c4490fef866d135bc27" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:06.819849 21379 raft_consensus.cc:2468] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:06.820465 21676 tablet_bootstrap.cc:492] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.821132 21420 leader_election.cc:304] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.822403 21674 raft_consensus.cc:2804] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.822773 21674 raft_consensus.cc:493] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.823132 21674 raft_consensus.cc:3060] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.823844 21677 tablet_bootstrap.cc:492] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: No bootstrap required, opened a new log
I20260709 12:15:06.824606 21677 ts_tablet_manager.cc:1403] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.002s
I20260709 12:15:06.825114 21673 leader_election.cc:290] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.828418 21677 raft_consensus.cc:359] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.829363 21677 raft_consensus.cc:385] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.829726 21677 raft_consensus.cc:740] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.830498 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d3292d44834c4490fef866d135bc27" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:06.830534 21677 consensus_queue.cc:260] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.831414 21527 raft_consensus.cc:2468] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 0.
I20260709 12:15:06.833290 21677 ts_tablet_manager.cc:1434] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:15:06.821152 21676 ts_tablet_manager.cc:1403] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.016s
I20260709 12:15:06.833623 21674 raft_consensus.cc:515] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.836694 21674 leader_election.cc:290] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.837360 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d3292d44834c4490fef866d135bc27" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:06.837486 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d3292d44834c4490fef866d135bc27" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:06.838052 21527 raft_consensus.cc:3060] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.838186 21379 raft_consensus.cc:3060] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.838114 21676 raft_consensus.cc:359] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.839339 21676 raft_consensus.cc:385] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.839643 21676 raft_consensus.cc:740] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.840431 21676 consensus_queue.cc:260] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.842722 21676 ts_tablet_manager.cc:1434] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:15:06.843782 21676 tablet_bootstrap.cc:492] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: Bootstrap starting.
I20260709 12:15:06.846876 21379 raft_consensus.cc:2468] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:06.848278 21672 raft_consensus.cc:493] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.848814 21672 raft_consensus.cc:515] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.850567 21676 tablet_bootstrap.cc:654] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:06.850726 21672 leader_election.cc:290] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.852207 21420 leader_election.cc:304] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.852500 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "697bb0a36e144873afd023990ca9660a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:06.852892 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "697bb0a36e144873afd023990ca9660a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.853312 21528 raft_consensus.cc:2468] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.853605 21453 raft_consensus.cc:2468] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.854754 21342 leader_election.cc:304] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.855716 21672 raft_consensus.cc:2804] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.856139 21672 raft_consensus.cc:493] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.856456 21672 raft_consensus.cc:3060] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.856894 21674 raft_consensus.cc:2804] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.857667 21674 raft_consensus.cc:697] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:06.858879 21674 consensus_queue.cc:237] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.864149 21671 raft_consensus.cc:493] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.864459 21672 raft_consensus.cc:515] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.864832 21671 raft_consensus.cc:515] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.866972 21671 leader_election.cc:290] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.868640 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "697bb0a36e144873afd023990ca9660a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.868809 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ffa400581a495dbb4717536e97c3b4" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.869354 21453 raft_consensus.cc:3060] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.869453 21452 raft_consensus.cc:2468] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.869694 21527 raft_consensus.cc:2468] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 1.
I20260709 12:15:06.870965 21344 leader_election.cc:304] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.876009 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ffa400581a495dbb4717536e97c3b4" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:06.877573 21671 raft_consensus.cc:2804] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.877926 21453 raft_consensus.cc:2468] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.878047 21671 raft_consensus.cc:493] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.878525 21671 raft_consensus.cc:3060] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.876808 21528 raft_consensus.cc:2468] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.887501 21671 raft_consensus.cc:515] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.890331 21672 leader_election.cc:290] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.892501 21344 leader_election.cc:304] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.894940 21671 leader_election.cc:290] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.895550 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "697bb0a36e144873afd023990ca9660a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:06.896346 21528 raft_consensus.cc:3060] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.897526 21676 tablet_bootstrap.cc:492] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: No bootstrap required, opened a new log
I20260709 12:15:06.896719 21276 catalog_manager.cc:5697] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:06.898145 21676 ts_tablet_manager.cc:1403] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.028s
I20260709 12:15:06.900367 21672 raft_consensus.cc:2804] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.901114 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ffa400581a495dbb4717536e97c3b4" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:06.901901 21527 raft_consensus.cc:3060] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.901556 21676 raft_consensus.cc:359] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.902345 21676 raft_consensus.cc:385] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:06.902628 21676 raft_consensus.cc:740] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Initialized, Role: FOLLOWER
I20260709 12:15:06.903448 21676 consensus_queue.cc:260] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.905810 21676 ts_tablet_manager.cc:1434] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:15:06.905844 21528 raft_consensus.cc:2468] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.907477 21691 consensus_queue.cc:1048] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:06.901194 21672 raft_consensus.cc:697] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:06.910149 21660 raft_consensus.cc:493] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.910903 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ffa400581a495dbb4717536e97c3b4" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.910774 21660 raft_consensus.cc:515] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.911604 21453 raft_consensus.cc:3060] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.911123 21672 consensus_queue.cc:237] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.913347 21660 leader_election.cc:290] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.916193 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.916965 21452 raft_consensus.cc:2468] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.918421 21344 leader_election.cc:304] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.918287 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:06.921650 21660 raft_consensus.cc:2804] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.922097 21660 raft_consensus.cc:493] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.921329 21528 raft_consensus.cc:2468] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 0.
I20260709 12:15:06.922783 21660 raft_consensus.cc:3060] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.922925 21527 raft_consensus.cc:2468] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.924429 21453 raft_consensus.cc:2468] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.925946 21342 leader_election.cc:304] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.926816 21671 raft_consensus.cc:2804] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.932477 21671 raft_consensus.cc:697] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:06.933673 21671 consensus_queue.cc:237] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.941293 21660 raft_consensus.cc:515] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.943790 21660 leader_election.cc:290] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:06.946128 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.946405 21453 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.946861 21452 raft_consensus.cc:3060] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.946765 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:06.949121 21528 raft_consensus.cc:3060] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.944644 21276 catalog_manager.cc:5697] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:06.958186 21691 consensus_queue.cc:1048] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:06.960160 21528 raft_consensus.cc:2468] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.961936 21342 leader_election.cc:304] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.963052 21660 raft_consensus.cc:2804] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:06.963815 21660 raft_consensus.cc:697] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:06.961910 21452 raft_consensus.cc:2468] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 1.
I20260709 12:15:06.966859 21691 raft_consensus.cc:493] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.966107 21660 consensus_queue.cc:237] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.967657 21691 raft_consensus.cc:515] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.971149 21691 leader_election.cc:290] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.971812 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6266bcfcf22444619a61bffac71edb32" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:06.971956 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6266bcfcf22444619a61bffac71edb32" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:06.972716 21379 raft_consensus.cc:2468] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:06.972716 21452 raft_consensus.cc:2468] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 0.
I20260709 12:15:06.973985 21494 leader_election.cc:304] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:06.975164 21691 raft_consensus.cc:2804] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:06.975351 21453 raft_consensus.cc:3060] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.975546 21691 raft_consensus.cc:493] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.976051 21691 raft_consensus.cc:3060] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.989734 21691 raft_consensus.cc:515] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:06.991029 21660 consensus_queue.cc:1048] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:06.995139 21691 leader_election.cc:290] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:06.995481 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6266bcfcf22444619a61bffac71edb32" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:06.996368 21453 raft_consensus.cc:3060] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:06.997053 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6266bcfcf22444619a61bffac71edb32" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:06.997726 21379 raft_consensus.cc:3060] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:07.000494 21276 catalog_manager.cc:5697] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.005000 21276 catalog_manager.cc:5697] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.006508 21453 raft_consensus.cc:2468] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:07.020234 21674 raft_consensus.cc:493] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ef39358b552d46568d4673bc03cb2e12)
I20260709 12:15:07.023797 21379 raft_consensus.cc:2468] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 1.
I20260709 12:15:07.020452 21671 consensus_queue.cc:1048] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:15:07.025638 21494 leader_election.cc:304] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:07.027848 21691 raft_consensus.cc:2804] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:07.028379 21691 raft_consensus.cc:697] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:07.024019 21674 raft_consensus.cc:515] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.029263 21691 consensus_queue.cc:237] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.040215 21674 leader_election.cc:290] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.041966 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:07.042200 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ef39358b552d46568d4673bc03cb2e12" is_pre_election: true
I20260709 12:15:07.047302 21417 leader_election.cc:304] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3; no voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:07.048620 21673 raft_consensus.cc:2749] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:15:07.055112 21276 catalog_manager.cc:5697] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:07.097299 21527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
mode: GRACEFUL
new_leader_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:42938
I20260709 12:15:07.098233 21527 raft_consensus.cc:606] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Received request to transfer leadership to TS ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:07.108219 21452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96d3292d44834c4490fef866d135bc27"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
mode: GRACEFUL
new_leader_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:58696
I20260709 12:15:07.108795 21452 raft_consensus.cc:606] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Received request to transfer leadership to TS ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:07.114284 21527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
mode: GRACEFUL
new_leader_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:42938
I20260709 12:15:07.114780 21527 raft_consensus.cc:606] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Received request to transfer leadership to TS ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:07.119760 21527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
mode: GRACEFUL
new_leader_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:42938
I20260709 12:15:07.120278 21527 raft_consensus.cc:606] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Received request to transfer leadership to TS aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:07.125735 21379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:54854
I20260709 12:15:07.126295 21379 raft_consensus.cc:606] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Received request to transfer leadership to TS aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:07.131089 21379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "697bb0a36e144873afd023990ca9660a"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:54854
I20260709 12:15:07.131556 21379 raft_consensus.cc:606] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Received request to transfer leadership to TS aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:07.133948 21671 raft_consensus.cc:493] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.134634 21671 raft_consensus.cc:515] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.136462 21379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:54854
I20260709 12:15:07.137110 21379 raft_consensus.cc:606] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Received request to transfer leadership to TS aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:07.137506 21671 leader_election.cc:290] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.137722 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa209a8923e14e68b1ba654178cedee3" is_pre_election: true
I20260709 12:15:07.138633 21452 raft_consensus.cc:2393] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef39358b552d46568d4673bc03cb2e12 in current term 1: Already voted for candidate a9ed2c75babe4e49a17ab052415d1434 in this term.
I20260709 12:15:07.138429 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ed2c75babe4e49a17ab052415d1434" is_pre_election: true
I20260709 12:15:07.139931 21342 leader_election.cc:304] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef39358b552d46568d4673bc03cb2e12; no voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3
I20260709 12:15:07.140736 21671 raft_consensus.cc:3060] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:07.148103 21671 raft_consensus.cc:2749] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:07.157490 21691 consensus_queue.cc:1048] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.169540 21655 consensus_queue.cc:1048] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:07.178604 21731 raft_consensus.cc:993] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: : Instructing follower ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:07.179188 21655 raft_consensus.cc:1081] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Signalling peer ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:07.180945 21379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:54846
I20260709 12:15:07.181493 21379 raft_consensus.cc:493] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.181787 21379 raft_consensus.cc:3060] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.188288 21379 raft_consensus.cc:515] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.190161 21379 leader_election.cc:290] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.191879 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:07.192310 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca7dfc01c7344a858d588558dcc56486" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.192682 21452 raft_consensus.cc:3060] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.193001 21527 raft_consensus.cc:3055] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.193324 21527 raft_consensus.cc:740] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:07.194211 21527 consensus_queue.cc:260] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.195752 21527 raft_consensus.cc:3060] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.201607 21452 raft_consensus.cc:2468] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:07.203140 21344 leader_election.cc:304] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:07.203946 21671 raft_consensus.cc:2804] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.204101 21527 raft_consensus.cc:2468] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:07.204422 21671 raft_consensus.cc:697] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:07.205513 21671 consensus_queue.cc:237] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.213407 21276 catalog_manager.cc:5697] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 1 to 2, leader changed from a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67) to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.244122 21673 consensus_queue.cc:1048] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:07.252115 21673 raft_consensus.cc:993] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: : Instructing follower ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:07.252789 21715 raft_consensus.cc:1081] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Signalling peer ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:07.255324 21377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96d3292d44834c4490fef866d135bc27"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:54848
I20260709 12:15:07.257200 21377 raft_consensus.cc:493] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.257614 21377 raft_consensus.cc:3060] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.259003 21673 consensus_queue.cc:1048] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.267813 21377 raft_consensus.cc:515] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.273223 21655 consensus_queue.cc:1048] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.276304 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d3292d44834c4490fef866d135bc27" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:07.277161 21454 raft_consensus.cc:3055] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.277487 21454 raft_consensus.cc:740] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:07.278483 21454 consensus_queue.cc:260] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.279675 21454 raft_consensus.cc:3060] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.284613 21731 raft_consensus.cc:993] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: : Instructing follower aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.285234 21691 raft_consensus.cc:1081] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Signalling peer aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.286063 21527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96d3292d44834c4490fef866d135bc27" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.286826 21527 raft_consensus.cc:3060] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.289592 21377 leader_election.cc:290] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.291042 21453 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:58686
I20260709 12:15:07.291692 21453 raft_consensus.cc:493] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.292080 21453 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.292676 21691 consensus_queue.cc:1048] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.298511 21454 raft_consensus.cc:2468] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:07.300020 21344 leader_election.cc:304] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:07.300885 21671 raft_consensus.cc:2804] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.301417 21671 raft_consensus.cc:697] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:07.301141 21453 raft_consensus.cc:515] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.302398 21452 raft_consensus.cc:1240] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Rejecting Update request from peer a9ed2c75babe4e49a17ab052415d1434 for earlier term 1. Current term is 2. Ops: []
I20260709 12:15:07.302506 21671 consensus_queue.cc:237] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.304668 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:07.307004 21691 consensus_queue.cc:1059] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.308228 21655 raft_consensus.cc:3055] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.308645 21655 raft_consensus.cc:740] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:07.309607 21655 consensus_queue.cc:260] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.310629 21655 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.312152 21453 leader_election.cc:290] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.312403 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.301409 21527 raft_consensus.cc:2468] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:07.326082 21671 consensus_queue.cc:1048] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.327031 21660 consensus_queue.cc:1048] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.332715 21528 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.336292 21417 leader_election.cc:304] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: ef39358b552d46568d4673bc03cb2e12
I20260709 12:15:07.336910 21275 catalog_manager.cc:5697] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 1 to 2, leader changed from aa209a8923e14e68b1ba654178cedee3 (127.9.11.66) to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.337663 21673 raft_consensus.cc:2804] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.344648 21673 raft_consensus.cc:697] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:07.345597 21673 consensus_queue.cc:237] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.349954 21672 consensus_queue.cc:1048] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.360320 21671 consensus_queue.cc:1048] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.362233 21274 catalog_manager.cc:5697] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 1 to 2, leader changed from a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67) to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.384661 21720 raft_consensus.cc:993] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: : Instructing follower aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.385233 21671 raft_consensus.cc:1081] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Signalling peer aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.387318 21452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "697bb0a36e144873afd023990ca9660a"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:58684
I20260709 12:15:07.387869 21452 raft_consensus.cc:493] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.388203 21452 raft_consensus.cc:3060] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.394758 21452 raft_consensus.cc:515] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.397081 21452 leader_election.cc:290] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.397922 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "697bb0a36e144873afd023990ca9660a" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:07.398684 21379 raft_consensus.cc:3055] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.399075 21379 raft_consensus.cc:740] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:07.399946 21379 consensus_queue.cc:260] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.399982 21525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "697bb0a36e144873afd023990ca9660a" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.401039 21525 raft_consensus.cc:3060] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.401417 21379 raft_consensus.cc:3060] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.402434 21671 consensus_queue.cc:1048] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.412173 21525 raft_consensus.cc:2468] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.418653 21379 raft_consensus.cc:2468] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.414095 21417 leader_election.cc:304] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:07.420125 21673 raft_consensus.cc:2804] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.420662 21673 raft_consensus.cc:697] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:07.421561 21673 consensus_queue.cc:237] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.422765 21720 consensus_queue.cc:1048] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.423656 21691 raft_consensus.cc:993] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: : Instructing follower ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:07.424213 21731 raft_consensus.cc:1081] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Signalling peer ef39358b552d46568d4673bc03cb2e12 to start an election
I20260709 12:15:07.426496 21379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
 from {username='slave'} at 127.0.0.1:54846
I20260709 12:15:07.427372 21379 raft_consensus.cc:493] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.427738 21379 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.432415 21275 catalog_manager.cc:5697] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 1 to 2, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.438832 21720 raft_consensus.cc:993] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: : Instructing follower aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.441021 21379 raft_consensus.cc:515] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.442662 21720 raft_consensus.cc:1081] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Signalling peer aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.445688 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.446377 21528 raft_consensus.cc:3055] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.446679 21528 raft_consensus.cc:740] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:07.447576 21528 consensus_queue.cc:260] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.448524 21452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:58684
I20260709 12:15:07.448850 21528 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.449229 21452 raft_consensus.cc:493] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.449683 21452 raft_consensus.cc:3060] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.451396 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "ef39358b552d46568d4673bc03cb2e12" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:07.452088 21453 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.445700 21379 leader_election.cc:290] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Requested vote from peers aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.456578 21720 raft_consensus.cc:993] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: : Instructing follower aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.458811 21731 consensus_queue.cc:1048] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:07.461457 21660 raft_consensus.cc:1081] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Signalling peer aa209a8923e14e68b1ba654178cedee3 to start an election
I20260709 12:15:07.463770 21451 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
 from {username='slave'} at 127.0.0.1:58684
I20260709 12:15:07.464583 21451 raft_consensus.cc:493] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:07.464944 21453 raft_consensus.cc:2468] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:07.465140 21451 raft_consensus.cc:3060] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.466213 21344 leader_election.cc:304] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:07.467262 21660 raft_consensus.cc:2804] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.458698 21452 raft_consensus.cc:515] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.468051 21660 raft_consensus.cc:697] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Leader. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:07.469322 21528 raft_consensus.cc:2468] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef39358b552d46568d4673bc03cb2e12 in term 2.
I20260709 12:15:07.470609 21452 leader_election.cc:290] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.471339 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:07.474958 21379 raft_consensus.cc:3055] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.472324 21660 consensus_queue.cc:237] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.473397 21525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf33fa7474b540eda2a042f357a5b947" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.476504 21525 raft_consensus.cc:3060] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.475332 21379 raft_consensus.cc:740] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:07.480499 21451 raft_consensus.cc:515] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.482622 21379 consensus_queue.cc:260] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.483350 21451 leader_election.cc:290] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67:35351)
I20260709 12:15:07.483984 21379 raft_consensus.cc:3060] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.485908 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:07.486590 21377 raft_consensus.cc:3055] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:07.486685 21528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b1c4f1a6724242be3bad15722432c2" candidate_uuid: "aa209a8923e14e68b1ba654178cedee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
I20260709 12:15:07.487012 21377 raft_consensus.cc:740] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:07.487439 21528 raft_consensus.cc:3060] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.487989 21377 consensus_queue.cc:260] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.489519 21377 raft_consensus.cc:3060] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.492055 21379 raft_consensus.cc:2468] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.486692 21276 catalog_manager.cc:5697] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 reported cstate change: term changed from 1 to 2, leader changed from a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67) to ef39358b552d46568d4673bc03cb2e12 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "ef39358b552d46568d4673bc03cb2e12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.494307 21655 consensus_queue.cc:1048] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:07.496568 21528 raft_consensus.cc:2468] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.496765 21420 leader_election.cc:304] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa209a8923e14e68b1ba654178cedee3, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:07.497879 21673 raft_consensus.cc:2804] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.498422 21673 raft_consensus.cc:697] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:07.499758 21673 consensus_queue.cc:237] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.500989 21417 leader_election.cc:304] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:07.495419 21525 raft_consensus.cc:2468] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.502307 21715 raft_consensus.cc:2804] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:07.502844 21715 raft_consensus.cc:697] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Leader. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:07.503975 21715 consensus_queue.cc:237] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:07.508148 21377 raft_consensus.cc:2468] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa209a8923e14e68b1ba654178cedee3 in term 2.
I20260709 12:15:07.518963 21275 catalog_manager.cc:5697] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 1 to 2, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.534111 21276 catalog_manager.cc:5697] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 reported cstate change: term changed from 1 to 2, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to aa209a8923e14e68b1ba654178cedee3 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "aa209a8923e14e68b1ba654178cedee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.700016 21454 raft_consensus.cc:1275] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.701570 21660 consensus_queue.cc:1048] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.709918 21525 raft_consensus.cc:1275] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.712706 21671 consensus_queue.cc:1048] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.758639 21528 raft_consensus.cc:1275] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.760188 21671 consensus_queue.cc:1048] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.769321 21454 raft_consensus.cc:1275] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.771152 21671 consensus_queue.cc:1048] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.829488 21378 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:07.836717 21378 raft_consensus.cc:1275] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.838387 21715 consensus_queue.cc:1048] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.847599 21528 raft_consensus.cc:1275] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.849895 21715 consensus_queue.cc:1048] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.861304 21454 raft_consensus.cc:1275] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.863075 21672 consensus_queue.cc:1048] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.871964 21528 raft_consensus.cc:1275] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer ef39358b552d46568d4673bc03cb2e12: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.877034 21660 consensus_queue.cc:1048] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.882217 21378 raft_consensus.cc:1275] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.885561 21715 consensus_queue.cc:1048] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:07.896447 21528 raft_consensus.cc:1275] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.898012 21674 consensus_queue.cc:1048] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.953871 21377 raft_consensus.cc:1275] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.955497 21673 consensus_queue.cc:1048] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.964941 21528 raft_consensus.cc:1275] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.966600 21674 consensus_queue.cc:1048] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.967772 21377 raft_consensus.cc:1275] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.969667 21715 consensus_queue.cc:1048] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:07.983232 21525 raft_consensus.cc:1275] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Refusing update from remote peer aa209a8923e14e68b1ba654178cedee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:07.985065 21715 consensus_queue.cc:1048] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:10.144901 21276 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59308:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:15:10.172794 21276 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59308:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:15:10.238507  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:10.341091 21766 tablet.cc:1724] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.355177 21768 tablet.cc:1724] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:15:10.457800 21405 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:ca7dfc01c7344a858d588558dcc56486)
W20260709 12:15:10.459332 21554 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:ca7dfc01c7344a858d588558dcc56486)
W20260709 12:15:10.469403 21554 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:ca7dfc01c7344a858d588558dcc56486)
W20260709 12:15:10.459651 21778 log.cc:927] Time spent T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:15:10.498404 21417 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:35351, user_credentials={real_user=slave}} blocked reactor thread for 58397.7us
W20260709 12:15:10.592463 21493 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:46681, user_credentials={real_user=slave}} blocked reactor thread for 44430.4us
I20260709 12:15:10.597240 21797 tablet.cc:1724] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.654663 21768 tablet.cc:1724] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:15:10.678188 21802 log.cc:927] Time spent T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 12:15:10.684556 21378 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:54864
I20260709 12:15:10.696688 21805 tablet.cc:1724] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.697141 21810 tablet.cc:1724] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.699909 21809 tablet.cc:1724] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.795882 21808 tablet.cc:1724] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.796895 21816 tablet.cc:1724] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.775430 21807 tablet.cc:1724] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.782779 21814 tablet.cc:1724] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.808923 21768 tablet.cc:1724] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.763931 21813 tablet.cc:1724] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.828090 21378 raft_consensus.cc:606] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Received request to transfer leadership to TS a9ed2c75babe4e49a17ab052415d1434
I20260709 12:15:10.853470 21809 tablet.cc:1724] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.856268 21766 tablet.cc:1724] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.861028 21810 tablet.cc:1724] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:15:10.865698 21342 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.67:35351, user_credentials={real_user=slave}} blocked reactor thread for 51053.4us
I20260709 12:15:10.870590 21776 raft_consensus.cc:993] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: : Instructing follower a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:10.871127 21776 raft_consensus.cc:1081] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Signalling peer a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:10.875876 21805 tablet.cc:1724] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.889958 21522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:42910
I20260709 12:15:10.890872 21522 raft_consensus.cc:493] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:10.891435 21522 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:10.892715 21807 tablet.cc:1724] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.846828 21819 tablet.cc:1724] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.891310 21805 tablet.cc:1724] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.904771 21522 raft_consensus.cc:515] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:10.908443 21375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:10.909179 21375 raft_consensus.cc:3055] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:10.909560 21375 raft_consensus.cc:740] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:10.910614 21375 consensus_queue.cc:260] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:10.912223 21375 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:10.916354 21807 tablet.cc:1724] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.920482 21522 leader_election.cc:290] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 3 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:10.846432 21797 tablet.cc:1724] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.924966 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a76cabcbbe0432eab77588d04b43a4a" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:10.928814 21454 raft_consensus.cc:3060] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:10.928221 21766 tablet.cc:1724] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.921211 21768 tablet.cc:1724] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.929173 21814 tablet.cc:1724] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.943817 21809 tablet.cc:1724] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.949613 21454 raft_consensus.cc:2468] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 3.
I20260709 12:15:10.959646 21816 tablet.cc:1724] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:10.980605 21493 leader_election.cc:304] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:10.984347 21803 raft_consensus.cc:2804] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:10.984947 21803 raft_consensus.cc:697] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 3 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:10.985853 21803 consensus_queue.cc:237] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:10.997839 21375 raft_consensus.cc:2468] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 3.
I20260709 12:15:11.009477 21454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
mode: GRACEFUL
new_leader_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:58702
I20260709 12:15:11.010215 21454 raft_consensus.cc:606] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Received request to transfer leadership to TS a9ed2c75babe4e49a17ab052415d1434
I20260709 12:15:11.014119  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:15:11.015121  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:11.034017 21274 catalog_manager.cc:5697] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 2 to 3, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:11.036064 21276 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6: soft_deleted_table [id=3f369596f04f46429cc1ad7dce1bffa9] alter complete (version 1)
I20260709 12:15:11.041774 21454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c"
dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
mode: GRACEFUL
new_leader_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:58702
I20260709 12:15:11.042465 21454 raft_consensus.cc:606] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Received request to transfer leadership to TS a9ed2c75babe4e49a17ab052415d1434
I20260709 12:15:11.046732 21375 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985"
dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
mode: GRACEFUL
new_leader_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:54864
I20260709 12:15:11.047435 21375 raft_consensus.cc:606] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Received request to transfer leadership to TS a9ed2c75babe4e49a17ab052415d1434
I20260709 12:15:11.049320  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:11.050093  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:11.131916 21785 raft_consensus.cc:993] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: : Instructing follower a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:11.132674 21792 raft_consensus.cc:1081] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Signalling peer a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:11.134406 21522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:15:11.135039 21522 raft_consensus.cc:493] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:11.135501 21522 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:11.142354 21522 raft_consensus.cc:515] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.144309 21522 leader_election.cc:290] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 3 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:11.145226 21375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:11.145566 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a907452714e4cfdaa3631724eea1969" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:11.145994 21375 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:11.146250 21454 raft_consensus.cc:3055] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:11.146629 21454 raft_consensus.cc:740] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:11.147501 21454 consensus_queue.cc:260] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.148622 21454 raft_consensus.cc:3060] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:11.152421 21375 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 3.
I20260709 12:15:11.153780 21494 leader_election.cc:304] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:11.154565 21803 raft_consensus.cc:2804] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:11.154731 21454 raft_consensus.cc:2468] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 3.
I20260709 12:15:11.155057 21803 raft_consensus.cc:697] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 3 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:11.156060 21803 consensus_queue.cc:237] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.163591 21276 catalog_manager.cc:5697] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 2 to 3, leader changed from aa209a8923e14e68b1ba654178cedee3 (127.9.11.66) to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:11.392678 21746 raft_consensus.cc:993] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: : Instructing follower a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:11.393244 21820 raft_consensus.cc:1081] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Signalling peer a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:11.394821 21522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:42910
I20260709 12:15:11.395418 21522 raft_consensus.cc:493] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:11.395740 21522 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:11.403040 21522 raft_consensus.cc:515] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.405117 21522 leader_election.cc:290] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 3 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:11.405903 21375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:11.406322 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f9a1ecaab34de6ade6f289f05c5985" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:11.406945 21375 raft_consensus.cc:3055] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:11.407136 21454 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:11.407454 21375 raft_consensus.cc:740] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ef39358b552d46568d4673bc03cb2e12, State: Running, Role: LEADER
I20260709 12:15:11.408588 21375 consensus_queue.cc:260] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.409919 21375 raft_consensus.cc:3060] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:11.414700 21454 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 3.
I20260709 12:15:11.416368 21493 leader_election.cc:304] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, aa209a8923e14e68b1ba654178cedee3; no voters: 
I20260709 12:15:11.417052 21785 raft_consensus.cc:993] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: : Instructing follower a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:11.417295 21803 raft_consensus.cc:2804] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:11.417701 21792 raft_consensus.cc:1081] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Signalling peer a9ed2c75babe4e49a17ab052415d1434 to start an election
I20260709 12:15:11.418038 21803 raft_consensus.cc:697] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 3 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:11.418658 21375 raft_consensus.cc:2468] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 3.
I20260709 12:15:11.419332 21803 consensus_queue.cc:237] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.420356 21522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c"
dest_uuid: "a9ed2c75babe4e49a17ab052415d1434"
 from {username='slave'} at 127.0.0.1:42924
I20260709 12:15:11.420936 21522 raft_consensus.cc:493] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:11.421363 21522 raft_consensus.cc:3060] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:11.431526 21522 raft_consensus.cc:515] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.431208 21276 catalog_manager.cc:5697] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 2 to 3, leader changed from ef39358b552d46568d4673bc03cb2e12 (127.9.11.65) to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:11.433593 21522 leader_election.cc:290] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 2 election: Requested vote from peers ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483), aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681)
I20260709 12:15:11.434559 21375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ef39358b552d46568d4673bc03cb2e12"
I20260709 12:15:11.434757 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75eb7b8d72c24d7f8950630e908abf8c" candidate_uuid: "a9ed2c75babe4e49a17ab052415d1434" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aa209a8923e14e68b1ba654178cedee3"
I20260709 12:15:11.435454 21375 raft_consensus.cc:3060] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:11.435461 21454 raft_consensus.cc:3055] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:11.436223 21454 raft_consensus.cc:740] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aa209a8923e14e68b1ba654178cedee3, State: Running, Role: LEADER
I20260709 12:15:11.437023 21454 consensus_queue.cc:260] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.438521 21454 raft_consensus.cc:3060] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:11.444233 21375 raft_consensus.cc:2468] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 2.
I20260709 12:15:11.445582 21494 leader_election.cc:304] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9ed2c75babe4e49a17ab052415d1434, ef39358b552d46568d4673bc03cb2e12; no voters: 
I20260709 12:15:11.445953 21454 raft_consensus.cc:2468] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9ed2c75babe4e49a17ab052415d1434 in term 2.
I20260709 12:15:11.446419 21803 raft_consensus.cc:2804] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:11.447049 21803 raft_consensus.cc:697] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 2 LEADER]: Becoming Leader. State: Replica: a9ed2c75babe4e49a17ab052415d1434, State: Running, Role: LEADER
I20260709 12:15:11.448038 21803 consensus_queue.cc:237] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } }
I20260709 12:15:11.462719 21276 catalog_manager.cc:5697] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 reported cstate change: term changed from 1 to 2, leader changed from aa209a8923e14e68b1ba654178cedee3 (127.9.11.66) to a9ed2c75babe4e49a17ab052415d1434 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "a9ed2c75babe4e49a17ab052415d1434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9ed2c75babe4e49a17ab052415d1434" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 35351 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:11.590212 21454 raft_consensus.cc:1275] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:15:11.591753 21803 consensus_queue.cc:1048] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:11.594503 21375 raft_consensus.cc:1275] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:15:11.597050 21803 consensus_queue.cc:1048] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:11.658408 21378 raft_consensus.cc:1275] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:15:11.659782 21801 consensus_queue.cc:1048] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:11.668587 21454 raft_consensus.cc:1275] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:15:11.670699 21827 consensus_queue.cc:1048] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:15:11.931792 21454 raft_consensus.cc:1275] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:11.933379 21801 consensus_queue.cc:1048] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:11.935163 21378 raft_consensus.cc:1275] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:11.936928 21801 consensus_queue.cc:1048] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:11.948266 21449 raft_consensus.cc:1275] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:11.950608 21801 consensus_queue.cc:1048] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa209a8923e14e68b1ba654178cedee3" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 46681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:11.979461 21378 raft_consensus.cc:1275] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Refusing update from remote peer a9ed2c75babe4e49a17ab052415d1434: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:11.981606 21827 consensus_queue.cc:1048] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef39358b552d46568d4673bc03cb2e12" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:12.051419  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:12.057067  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:12.057494  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:12.063920  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:12.064383  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:13.065905  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:13.073146  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:13.073705  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:13.080111  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:13.080582  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:14.081820  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:14.087978  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:14.088536  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:14.094405  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:14.094894  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:15.096181  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:15.102315  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:15.102895  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:15.107970  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:15.108513  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:16.110404  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:16.120796  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:16.121400  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:16.127739  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:16.128221  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:17.129796  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:17.136682  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:17.137168  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:17.141937  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:17.142356  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:18.143728  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:18.149327  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:18.149825  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:18.155985  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:18.156575  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:19.157929  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:19.163318  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:19.163834  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:19.169114  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:19.169546  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:20.170676  9261 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:15:20.177475  9261 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:15:20.178161  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:20.184669  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:20.185276  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:21.252401  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
W20260709 12:15:21.265834 21420 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 12:15:21.272735 21420 consensus_peers.cc:597] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:21.298327 21494 consensus_peers.cc:597] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.308341  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:21.309367  9261 tablet_replica.cc:333] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.311271  9261 raft_consensus.cc:2243] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.312500  9261 raft_consensus.cc:2272] T 6266bcfcf22444619a61bffac71edb32 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.317306  9261 tablet_replica.cc:333] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.318414  9261 raft_consensus.cc:2243] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:15:21.319519 21420 consensus_peers.cc:597] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.320027  9261 raft_consensus.cc:2272] T 11ffa400581a495dbb4717536e97c3b4 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.323457  9261 tablet_replica.cc:333] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.324244  9261 raft_consensus.cc:2243] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.325429  9261 raft_consensus.cc:2272] T de0f5c45176b4a82a53549389e30b72e P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.328495  9261 tablet_replica.cc:333] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.329227  9261 raft_consensus.cc:2243] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.329836  9261 raft_consensus.cc:2272] T 93d74552227943f4afdee2a1ea3c547d P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.333616  9261 tablet_replica.cc:333] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.334342  9261 raft_consensus.cc:2243] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.335814  9261 raft_consensus.cc:2272] T ca7dfc01c7344a858d588558dcc56486 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.339440  9261 tablet_replica.cc:333] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.340390  9261 raft_consensus.cc:2243] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.341050  9261 raft_consensus.cc:2272] T d90860574723485e8d764982f9abbf13 P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.343691  9261 tablet_replica.cc:333] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.344295  9261 raft_consensus.cc:2243] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.345003  9261 raft_consensus.cc:2272] T 4a76cabcbbe0432eab77588d04b43a4a P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.347895  9261 tablet_replica.cc:333] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.348600  9261 raft_consensus.cc:2243] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.349647  9261 raft_consensus.cc:2272] T 8ca48b558d1c438c868aa42cdff43269 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.352502  9261 tablet_replica.cc:333] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.353202  9261 raft_consensus.cc:2243] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.353866  9261 raft_consensus.cc:2272] T 78b1c4f1a6724242be3bad15722432c2 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.356662  9261 tablet_replica.cc:333] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.357263  9261 raft_consensus.cc:2243] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.357848  9261 raft_consensus.cc:2272] T 80a0c0185f6e4e9193531352768d711a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.360563  9261 tablet_replica.cc:333] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.361178  9261 raft_consensus.cc:2243] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.361897  9261 raft_consensus.cc:2272] T 8a907452714e4cfdaa3631724eea1969 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.364755  9261 tablet_replica.cc:333] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.365593  9261 raft_consensus.cc:2243] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.366546  9261 raft_consensus.cc:2272] T bf33fa7474b540eda2a042f357a5b947 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.370141  9261 tablet_replica.cc:333] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.370760  9261 raft_consensus.cc:2243] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.371414  9261 raft_consensus.cc:2272] T 697bb0a36e144873afd023990ca9660a P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.374122  9261 tablet_replica.cc:333] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.374788  9261 raft_consensus.cc:2243] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.375399  9261 raft_consensus.cc:2272] T d4f9a1ecaab34de6ade6f289f05c5985 P ef39358b552d46568d4673bc03cb2e12 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.378473  9261 tablet_replica.cc:333] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.379159  9261 raft_consensus.cc:2243] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.379794  9261 raft_consensus.cc:2272] T 002b11df78754235ae83eb63aaebe2c9 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.383765  9261 tablet_replica.cc:333] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.384462  9261 raft_consensus.cc:2243] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.385660  9261 raft_consensus.cc:2272] T 96d3292d44834c4490fef866d135bc27 P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.388741  9261 tablet_replica.cc:333] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.389338  9261 raft_consensus.cc:2243] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.389886  9261 raft_consensus.cc:2272] T 75eb7b8d72c24d7f8950630e908abf8c P ef39358b552d46568d4673bc03cb2e12 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.392963  9261 tablet_replica.cc:333] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12: stopping tablet replica
I20260709 12:15:21.393512  9261 raft_consensus.cc:2243] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.394603  9261 raft_consensus.cc:2272] T 88e983ca86ec4bf795b7e5fc727b9def P ef39358b552d46568d4673bc03cb2e12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.450958  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
I20260709 12:15:21.514546  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
I20260709 12:15:21.555796  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:21.556810  9261 tablet_replica.cc:333] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.558249  9261 raft_consensus.cc:2243] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.559237  9261 raft_consensus.cc:2272] T 6266bcfcf22444619a61bffac71edb32 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.562402  9261 tablet_replica.cc:333] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.563144  9261 raft_consensus.cc:2243] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.563760  9261 raft_consensus.cc:2272] T 11ffa400581a495dbb4717536e97c3b4 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.566541  9261 tablet_replica.cc:333] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.567335  9261 raft_consensus.cc:2243] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.568351  9261 raft_consensus.cc:2272] T bf33fa7474b540eda2a042f357a5b947 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.571295  9261 tablet_replica.cc:333] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.572187  9261 raft_consensus.cc:2243] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.572814  9261 raft_consensus.cc:2272] T 8a907452714e4cfdaa3631724eea1969 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.576455  9261 tablet_replica.cc:333] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.577463  9261 raft_consensus.cc:2243] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.578207  9261 raft_consensus.cc:2272] T 75eb7b8d72c24d7f8950630e908abf8c P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:21.578676 21494 consensus_peers.cc:597] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:40483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:21.581054 21494 consensus_peers.cc:597] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:40483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.581453  9261 tablet_replica.cc:333] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.582274  9261 raft_consensus.cc:2243] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.582902  9261 raft_consensus.cc:2272] T 96d3292d44834c4490fef866d135bc27 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.585896  9261 tablet_replica.cc:333] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.586593  9261 raft_consensus.cc:2243] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.587206  9261 raft_consensus.cc:2272] T 88e983ca86ec4bf795b7e5fc727b9def P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.590006  9261 tablet_replica.cc:333] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.590802  9261 raft_consensus.cc:2243] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.591843  9261 raft_consensus.cc:2272] T 78b1c4f1a6724242be3bad15722432c2 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.594391  9261 tablet_replica.cc:333] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.595029  9261 raft_consensus.cc:2243] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.596091  9261 raft_consensus.cc:2272] T 80a0c0185f6e4e9193531352768d711a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.598569  9261 tablet_replica.cc:333] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.599258  9261 raft_consensus.cc:2243] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.599807  9261 raft_consensus.cc:2272] T de0f5c45176b4a82a53549389e30b72e P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.602797  9261 tablet_replica.cc:333] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.603444  9261 raft_consensus.cc:2243] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.604028  9261 raft_consensus.cc:2272] T 93d74552227943f4afdee2a1ea3c547d P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.606467  9261 tablet_replica.cc:333] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.607100  9261 raft_consensus.cc:2243] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.607649  9261 raft_consensus.cc:2272] T 4a76cabcbbe0432eab77588d04b43a4a P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.610407  9261 tablet_replica.cc:333] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.611056  9261 raft_consensus.cc:2243] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.611635  9261 raft_consensus.cc:2272] T 8ca48b558d1c438c868aa42cdff43269 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.614359  9261 tablet_replica.cc:333] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.614998  9261 raft_consensus.cc:2243] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.615573  9261 raft_consensus.cc:2272] T ca7dfc01c7344a858d588558dcc56486 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.618238  9261 tablet_replica.cc:333] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.618814  9261 raft_consensus.cc:2243] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.619861  9261 raft_consensus.cc:2272] T d90860574723485e8d764982f9abbf13 P aa209a8923e14e68b1ba654178cedee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.622587  9261 tablet_replica.cc:333] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.623239  9261 raft_consensus.cc:2243] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.624313  9261 raft_consensus.cc:2272] T 697bb0a36e144873afd023990ca9660a P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.626834  9261 tablet_replica.cc:333] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.627408  9261 raft_consensus.cc:2243] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.628008  9261 raft_consensus.cc:2272] T d4f9a1ecaab34de6ade6f289f05c5985 P aa209a8923e14e68b1ba654178cedee3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.630584  9261 tablet_replica.cc:333] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3: stopping tablet replica
I20260709 12:15:21.631132  9261 raft_consensus.cc:2243] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.632169  9261 raft_consensus.cc:2272] T 002b11df78754235ae83eb63aaebe2c9 P aa209a8923e14e68b1ba654178cedee3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:21.650168 21494 consensus_peers.cc:597] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:40483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.685022  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
W20260709 12:15:21.713328 21493 consensus_peers.cc:597] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 -> Peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Couldn't send request to peer aa209a8923e14e68b1ba654178cedee3. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:46681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:21.729210 21493 consensus_peers.cc:597] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 -> Peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Couldn't send request to peer aa209a8923e14e68b1ba654178cedee3. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:46681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.746912  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
W20260709 12:15:21.748812 21494 consensus_peers.cc:597] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:40483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:21.777495 21494 consensus_peers.cc:597] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 -> Peer ef39358b552d46568d4673bc03cb2e12 (127.9.11.65:40483): Couldn't send request to peer ef39358b552d46568d4673bc03cb2e12. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:40483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.792884  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:21.793818  9261 tablet_replica.cc:333] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.794742  9261 raft_consensus.cc:2243] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.795895  9261 raft_consensus.cc:2272] T 6266bcfcf22444619a61bffac71edb32 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.798840  9261 tablet_replica.cc:333] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.799564  9261 raft_consensus.cc:2243] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.800144  9261 raft_consensus.cc:2272] T 11ffa400581a495dbb4717536e97c3b4 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.802613  9261 tablet_replica.cc:333] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.803365  9261 raft_consensus.cc:2243] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.803941  9261 raft_consensus.cc:2272] T ca7dfc01c7344a858d588558dcc56486 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.806504  9261 tablet_replica.cc:333] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.807222  9261 raft_consensus.cc:2243] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.807746  9261 raft_consensus.cc:2272] T 8ca48b558d1c438c868aa42cdff43269 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.810230  9261 tablet_replica.cc:333] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.810817  9261 raft_consensus.cc:2243] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.812016  9261 raft_consensus.cc:2272] T 4a76cabcbbe0432eab77588d04b43a4a P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.815228  9261 tablet_replica.cc:333] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.816211  9261 raft_consensus.cc:2243] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.816902  9261 raft_consensus.cc:2272] T d90860574723485e8d764982f9abbf13 P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.820312  9261 tablet_replica.cc:333] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.821175  9261 raft_consensus.cc:2243] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.821868  9261 raft_consensus.cc:2272] T 96d3292d44834c4490fef866d135bc27 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.825043  9261 tablet_replica.cc:333] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.825644  9261 raft_consensus.cc:2243] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.826627  9261 raft_consensus.cc:2272] T 75eb7b8d72c24d7f8950630e908abf8c P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.828991  9261 tablet_replica.cc:333] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.829643  9261 raft_consensus.cc:2243] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.830157  9261 raft_consensus.cc:2272] T de0f5c45176b4a82a53549389e30b72e P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.832849  9261 tablet_replica.cc:333] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.833446  9261 raft_consensus.cc:2243] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.834662  9261 raft_consensus.cc:2272] T 93d74552227943f4afdee2a1ea3c547d P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.837857  9261 tablet_replica.cc:333] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
W20260709 12:15:21.837759 21493 consensus_peers.cc:597] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 -> Peer aa209a8923e14e68b1ba654178cedee3 (127.9.11.66:46681): Couldn't send request to peer aa209a8923e14e68b1ba654178cedee3. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:46681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:21.838972  9261 raft_consensus.cc:2243] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.839586  9261 raft_consensus.cc:2272] T 78b1c4f1a6724242be3bad15722432c2 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.841979  9261 tablet_replica.cc:333] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.842540  9261 raft_consensus.cc:2243] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.843056  9261 raft_consensus.cc:2272] T 80a0c0185f6e4e9193531352768d711a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.845418  9261 tablet_replica.cc:333] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.845925  9261 raft_consensus.cc:2243] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.846326  9261 raft_consensus.cc:2272] T 88e983ca86ec4bf795b7e5fc727b9def P a9ed2c75babe4e49a17ab052415d1434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.848564  9261 tablet_replica.cc:333] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.849052  9261 raft_consensus.cc:2243] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.850024  9261 raft_consensus.cc:2272] T 8a907452714e4cfdaa3631724eea1969 P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.852509  9261 tablet_replica.cc:333] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.853063  9261 raft_consensus.cc:2243] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.853590  9261 raft_consensus.cc:2272] T bf33fa7474b540eda2a042f357a5b947 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.855978  9261 tablet_replica.cc:333] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.856506  9261 raft_consensus.cc:2243] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.857028  9261 raft_consensus.cc:2272] T 697bb0a36e144873afd023990ca9660a P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.859400  9261 tablet_replica.cc:333] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.859902  9261 raft_consensus.cc:2243] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:21.860816  9261 raft_consensus.cc:2272] T d4f9a1ecaab34de6ade6f289f05c5985 P a9ed2c75babe4e49a17ab052415d1434 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.863360  9261 tablet_replica.cc:333] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434: stopping tablet replica
I20260709 12:15:21.863852  9261 raft_consensus.cc:2243] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:21.864288  9261 raft_consensus.cc:2272] T 002b11df78754235ae83eb63aaebe2c9 P a9ed2c75babe4e49a17ab052415d1434 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:21.916136  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:15:21.966360  9261 master.cc:562] Master@127.9.11.126:37351 shutting down...
I20260709 12:15:22.000607  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:22.001394  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:22.001791  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8a70a01ac6a445b7942a6858b54cbed6: stopping tablet replica
I20260709 12:15:22.026275  9261 master.cc:584] Master@127.9.11.126:37351 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21254 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:15:22.093665  9261 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.11.126:34477
I20260709 12:15:22.095261  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:22.101967 21867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.101982 21868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.104632 21870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:22.105744  9261 server_base.cc:1061] running on GCE node
I20260709 12:15:22.107249  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:22.107461  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:22.107666  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599322107626 us; error 0 us; skew 500 ppm
I20260709 12:15:22.108278  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:22.111719  9261 webserver.cc:533] Webserver started at http://127.9.11.126:37659/ using document root <none> and password file <none>
I20260709 12:15:22.112241  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:22.112428  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:22.112674  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:22.114094  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/master-0-root/instance:
uuid: "8d4023a6dc134583a76269f52824ceac"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.119448  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:15:22.123502 21875 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.124380  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:15:22.124732  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/master-0-root
uuid: "8d4023a6dc134583a76269f52824ceac"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.125039  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:22.150046  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:22.151721  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:22.198781  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.126:34477
I20260709 12:15:22.198918 21926 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.126:34477 every 8 connection(s)
I20260709 12:15:22.204237 21927 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.218169 21927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac: Bootstrap starting.
I20260709 12:15:22.224234 21927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:22.229585 21927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac: No bootstrap required, opened a new log
I20260709 12:15:22.232411 21927 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4023a6dc134583a76269f52824ceac" member_type: VOTER }
I20260709 12:15:22.232955 21927 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:22.233203 21927 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d4023a6dc134583a76269f52824ceac, State: Initialized, Role: FOLLOWER
I20260709 12:15:22.234089 21927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4023a6dc134583a76269f52824ceac" member_type: VOTER }
I20260709 12:15:22.234651 21927 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:22.234962 21927 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:22.235289 21927 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:22.241879 21927 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4023a6dc134583a76269f52824ceac" member_type: VOTER }
I20260709 12:15:22.242542 21927 leader_election.cc:304] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d4023a6dc134583a76269f52824ceac; no voters: 
I20260709 12:15:22.244093 21927 leader_election.cc:290] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:22.244699 21930 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:22.246373 21930 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 1 LEADER]: Becoming Leader. State: Replica: 8d4023a6dc134583a76269f52824ceac, State: Running, Role: LEADER
I20260709 12:15:22.247272 21930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4023a6dc134583a76269f52824ceac" member_type: VOTER }
I20260709 12:15:22.247788 21927 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:22.254460 21931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d4023a6dc134583a76269f52824ceac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4023a6dc134583a76269f52824ceac" member_type: VOTER } }
I20260709 12:15:22.254482 21932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d4023a6dc134583a76269f52824ceac. Latest consensus state: current_term: 1 leader_uuid: "8d4023a6dc134583a76269f52824ceac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d4023a6dc134583a76269f52824ceac" member_type: VOTER } }
I20260709 12:15:22.255553 21931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:22.255573 21932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:22.263676 21938 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:22.269838 21938 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:22.271186  9261 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:22.280620 21938 catalog_manager.cc:1383] Generated new cluster ID: fc322d80700d4ede9e977077c3449aad
I20260709 12:15:22.280961 21938 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:22.307070 21938 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:22.308645 21938 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:22.324769 21938 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac: Generated new TSK 0
I20260709 12:15:22.325517 21938 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:22.338791  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:22.346163 21948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.347551 21949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:22.353318  9261 server_base.cc:1061] running on GCE node
W20260709 12:15:22.354203 21951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:22.355345  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:22.355590  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:22.355777  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599322355746 us; error 0 us; skew 500 ppm
I20260709 12:15:22.356334  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:22.358831  9261 webserver.cc:533] Webserver started at http://127.9.11.65:37027/ using document root <none> and password file <none>
I20260709 12:15:22.359369  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:22.359601  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:22.359902  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:22.361202  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-0-root/instance:
uuid: "e2a89f53d77e4029be05d7a9d2172586"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.366438  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:15:22.370304 21956 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.371199  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:15:22.371510  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-0-root
uuid: "e2a89f53d77e4029be05d7a9d2172586"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.371860  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:22.393913  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:22.395539  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:22.397277  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:22.399848  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:22.400087  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.400386  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:22.400548  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.450147  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.65:46861
I20260709 12:15:22.450273 22018 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.65:46861 every 8 connection(s)
I20260709 12:15:22.456077  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:22.464357 22023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.466421 22024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.468713 22026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:22.469524  9261 server_base.cc:1061] running on GCE node
I20260709 12:15:22.471627  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:22.471889  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:22.472076  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599322472060 us; error 0 us; skew 500 ppm
I20260709 12:15:22.472668  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:22.477667 22019 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34477
I20260709 12:15:22.478103 22019 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:22.479023 22019 heartbeater.cc:507] Master 127.9.11.126:34477 requested a full tablet report, sending...
I20260709 12:15:22.479667  9261 webserver.cc:533] Webserver started at http://127.9.11.66:39669/ using document root <none> and password file <none>
I20260709 12:15:22.480338  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:22.480568  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:22.480916  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:22.481734 21892 ts_manager.cc:194] Registered new tserver with Master: e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861)
I20260709 12:15:22.482957  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-1-root/instance:
uuid: "738c8826f8f5430d937cdbb7805bdfb8"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.484894 21892 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38956
I20260709 12:15:22.489277  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:15:22.493636 22031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.494496  9261 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:15:22.494831  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-1-root
uuid: "738c8826f8f5430d937cdbb7805bdfb8"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.495254  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:22.520838  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:22.522346  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:22.524112  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:22.526875  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:22.527151  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:15:22.527362  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:22.527489  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.575788  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.66:44195
I20260709 12:15:22.575928 22093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.66:44195 every 8 connection(s)
I20260709 12:15:22.581287  9261 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:22.589799 22097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.591472 22098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:22.594722 22101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:22.595279 22094 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34477
I20260709 12:15:22.595357  9261 server_base.cc:1061] running on GCE node
I20260709 12:15:22.596206 22094 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:22.596961  9261 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:22.597267  9261 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:22.597500  9261 hybrid_clock.cc:648] HybridClock initialized: now 1783599322597485 us; error 0 us; skew 500 ppm
I20260709 12:15:22.597325 22094 heartbeater.cc:507] Master 127.9.11.126:34477 requested a full tablet report, sending...
I20260709 12:15:22.598277  9261 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:22.600106 21892 ts_manager.cc:194] Registered new tserver with Master: 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:22.601189  9261 webserver.cc:533] Webserver started at http://127.9.11.67:33107/ using document root <none> and password file <none>
I20260709 12:15:22.601794  9261 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:22.602022  9261 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:22.601915 21892 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38960
I20260709 12:15:22.602512  9261 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:22.604108  9261 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-2-root/instance:
uuid: "8600e33421784ecb83a556ef90148dd3"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.609063  9261 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:22.612831 22105 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.613694  9261 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:15:22.614055  9261 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-2-root
uuid: "8600e33421784ecb83a556ef90148dd3"
format_stamp: "Formatted at 2026-07-09 12:15:22 on dist-test-slave-k289"
I20260709 12:15:22.614374  9261 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskRG0vXJ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598894027500-9261-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:22.628687  9261 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:22.630133  9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:22.631628  9261 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:22.634078  9261 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:22.634344  9261 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:22.634606  9261 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:22.635025  9261 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:15:22.683406  9261 rpc_server.cc:307] RPC server started. Bound to: 127.9.11.67:33391
I20260709 12:15:22.683535 22167 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.9.11.67:33391 every 8 connection(s)
I20260709 12:15:22.699527 22168 heartbeater.cc:344] Connected to a master server at 127.9.11.126:34477
I20260709 12:15:22.700066 22168 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:22.700911 22168 heartbeater.cc:507] Master 127.9.11.126:34477 requested a full tablet report, sending...
I20260709 12:15:22.703051 21892 ts_manager.cc:194] Registered new tserver with Master: 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:22.703214  9261 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016203376s
I20260709 12:15:22.704648 21892 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38964
I20260709 12:15:22.732147 21892 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38974:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:22.735396 21892 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:22.808980 21984 tablet_service.cc:1511] Processing CreateTablet for tablet 2339e13a2b06478b8e08f568b25762a1 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:22.809779 21983 tablet_service.cc:1511] Processing CreateTablet for tablet 4cbb5d889cc3467e9a166982b4eacfe0 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:22.810868 21984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2339e13a2b06478b8e08f568b25762a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.811775 21983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cbb5d889cc3467e9a166982b4eacfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.813436 21982 tablet_service.cc:1511] Processing CreateTablet for tablet 1fcbeb90ec3641b88309d64a33a7fe90 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:22.815021 21982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fcbeb90ec3641b88309d64a33a7fe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.819612 21981 tablet_service.cc:1511] Processing CreateTablet for tablet a19e3221c5de4c38802bdebb3994c029 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:22.821187 21981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a19e3221c5de4c38802bdebb3994c029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.826601 22059 tablet_service.cc:1511] Processing CreateTablet for tablet 2339e13a2b06478b8e08f568b25762a1 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:22.828209 22059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2339e13a2b06478b8e08f568b25762a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.829819 22058 tablet_service.cc:1511] Processing CreateTablet for tablet 4cbb5d889cc3467e9a166982b4eacfe0 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:22.831372 22058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cbb5d889cc3467e9a166982b4eacfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.832214 22057 tablet_service.cc:1511] Processing CreateTablet for tablet 1fcbeb90ec3641b88309d64a33a7fe90 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:22.833783 22057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fcbeb90ec3641b88309d64a33a7fe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.845317 22133 tablet_service.cc:1511] Processing CreateTablet for tablet 2339e13a2b06478b8e08f568b25762a1 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:22.847139 22133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2339e13a2b06478b8e08f568b25762a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.850097 21980 tablet_service.cc:1511] Processing CreateTablet for tablet 2f55e2d5913649e6a00e298a453cc96a (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:22.851773 21980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f55e2d5913649e6a00e298a453cc96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.858011 21979 tablet_service.cc:1511] Processing CreateTablet for tablet 673096df60ed44bc8b2a9856e6e1b884 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:22.859673 21979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673096df60ed44bc8b2a9856e6e1b884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.870180 22056 tablet_service.cc:1511] Processing CreateTablet for tablet a19e3221c5de4c38802bdebb3994c029 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:22.874133 22056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a19e3221c5de4c38802bdebb3994c029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.877215 22055 tablet_service.cc:1511] Processing CreateTablet for tablet 2f55e2d5913649e6a00e298a453cc96a (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:22.877681 22054 tablet_service.cc:1511] Processing CreateTablet for tablet 673096df60ed44bc8b2a9856e6e1b884 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:22.877231 22132 tablet_service.cc:1511] Processing CreateTablet for tablet 4cbb5d889cc3467e9a166982b4eacfe0 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:22.880788 22132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cbb5d889cc3467e9a166982b4eacfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.882251 22054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673096df60ed44bc8b2a9856e6e1b884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.883282 22055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f55e2d5913649e6a00e298a453cc96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.926582 22183 tablet_bootstrap.cc:492] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:22.931428 21978 tablet_service.cc:1511] Processing CreateTablet for tablet 8f84307224ba4c489d4120bc91039178 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:22.933221 21978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f84307224ba4c489d4120bc91039178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.936242 22131 tablet_service.cc:1511] Processing CreateTablet for tablet 1fcbeb90ec3641b88309d64a33a7fe90 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:22.937861 22131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fcbeb90ec3641b88309d64a33a7fe90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.941560 22058 tablet_service.cc:1511] Processing CreateTablet for tablet 8f84307224ba4c489d4120bc91039178 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:22.943276 22058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f84307224ba4c489d4120bc91039178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.953219 22130 tablet_service.cc:1511] Processing CreateTablet for tablet a19e3221c5de4c38802bdebb3994c029 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:22.955027 22128 tablet_service.cc:1511] Processing CreateTablet for tablet 673096df60ed44bc8b2a9856e6e1b884 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:22.953210 22129 tablet_service.cc:1511] Processing CreateTablet for tablet 2f55e2d5913649e6a00e298a453cc96a (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:22.960631 22130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a19e3221c5de4c38802bdebb3994c029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.962260 22129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f55e2d5913649e6a00e298a453cc96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:22.964332 22128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673096df60ed44bc8b2a9856e6e1b884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.079748 22054 tablet_service.cc:1511] Processing CreateTablet for tablet 8c965fb2a6f24b2eace0008b8f83bbbb (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:23.080592 21977 tablet_service.cc:1511] Processing CreateTablet for tablet 8c965fb2a6f24b2eace0008b8f83bbbb (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:23.081636 22054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c965fb2a6f24b2eace0008b8f83bbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.082187 21977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c965fb2a6f24b2eace0008b8f83bbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.110687 22127 tablet_service.cc:1511] Processing CreateTablet for tablet 8f84307224ba4c489d4120bc91039178 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:23.097681 22183 tablet_bootstrap.cc:654] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.116243 22125 tablet_service.cc:1511] Processing CreateTablet for tablet 335b3b26e87f404fa95b6d2a3c85f605 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:23.116262 22126 tablet_service.cc:1511] Processing CreateTablet for tablet 8c965fb2a6f24b2eace0008b8f83bbbb (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:23.118001 22127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f84307224ba4c489d4120bc91039178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.119412 22125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 335b3b26e87f404fa95b6d2a3c85f605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.120188 22126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c965fb2a6f24b2eace0008b8f83bbbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.128192 21982 tablet_service.cc:1511] Processing CreateTablet for tablet 335b3b26e87f404fa95b6d2a3c85f605 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:23.129864 21982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 335b3b26e87f404fa95b6d2a3c85f605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.098950 22058 tablet_service.cc:1511] Processing CreateTablet for tablet 335b3b26e87f404fa95b6d2a3c85f605 (DEFAULT_TABLE table=test-workload [id=37994f5f707448b19f7d7ebee7e1e765]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:23.132236 22058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 335b3b26e87f404fa95b6d2a3c85f605. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:23.169507 22184 tablet_bootstrap.cc:492] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.194592 22184 tablet_bootstrap.cc:654] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.200536 22185 tablet_bootstrap.cc:492] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.220507 22183 tablet_bootstrap.cc:492] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.223173 22183 ts_tablet_manager.cc:1403] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.297s	user 0.020s	sys 0.021s
I20260709 12:15:23.224006 22185 tablet_bootstrap.cc:654] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.227420 22184 tablet_bootstrap.cc:492] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.227939 22184 ts_tablet_manager.cc:1403] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.006s
I20260709 12:15:23.228706 22183 raft_consensus.cc:359] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.229882 22183 raft_consensus.cc:385] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.230266 22183 raft_consensus.cc:740] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.230926 22184 raft_consensus.cc:359] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.231709 22184 raft_consensus.cc:385] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.231220 22183 consensus_queue.cc:260] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.231992 22184 raft_consensus.cc:740] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.232798 22184 consensus_queue.cc:260] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.233094 22185 tablet_bootstrap.cc:492] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.233906 22185 ts_tablet_manager.cc:1403] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.034s	user 0.004s	sys 0.010s
I20260709 12:15:23.235600 22184 ts_tablet_manager.cc:1434] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:15:23.235410 22019 heartbeater.cc:499] Master 127.9.11.126:34477 was elected leader, sending a full tablet report...
I20260709 12:15:23.236735 22184 tablet_bootstrap.cc:492] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.239955 22094 heartbeater.cc:499] Master 127.9.11.126:34477 was elected leader, sending a full tablet report...
I20260709 12:15:23.241055 22183 ts_tablet_manager.cc:1434] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.005s
I20260709 12:15:23.242269 22183 tablet_bootstrap.cc:492] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.243224 22184 tablet_bootstrap.cc:654] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.250176 22183 tablet_bootstrap.cc:654] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.254699 22185 raft_consensus.cc:359] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.255725 22185 raft_consensus.cc:385] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.255636 22189 raft_consensus.cc:493] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.256569 22189 raft_consensus.cc:515] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.257505 22184 tablet_bootstrap.cc:492] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.258183 22184 ts_tablet_manager.cc:1403] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.008s
I20260709 12:15:23.259115 22185 raft_consensus.cc:740] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.260151 22189 leader_election.cc:290] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.260030 22185 consensus_queue.cc:260] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.265300 22183 tablet_bootstrap.cc:492] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.266326 22183 ts_tablet_manager.cc:1403] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.000s
W20260709 12:15:23.266386 21940 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:23.273512 22183 raft_consensus.cc:359] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.274669 22183 raft_consensus.cc:385] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.275112 22183 raft_consensus.cc:740] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.274669 22184 raft_consensus.cc:359] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.275794 22184 raft_consensus.cc:385] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.276124 22184 raft_consensus.cc:740] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.277956 22184 consensus_queue.cc:260] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.277868 22183 consensus_queue.cc:260] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.281383 22184 ts_tablet_manager.cc:1434] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.001s
I20260709 12:15:23.287273 22184 tablet_bootstrap.cc:492] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.290715 22183 ts_tablet_manager.cc:1434] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.003s
I20260709 12:15:23.292613 22183 tablet_bootstrap.cc:492] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.295727 22184 tablet_bootstrap.cc:654] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.296048 22190 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.306340 22190 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.311450 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.313534 22190 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
W20260709 12:15:23.316272 22032 leader_election.cc:343] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.317620 22184 tablet_bootstrap.cc:492] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.318320 22184 ts_tablet_manager.cc:1403] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.007s
I20260709 12:15:23.311700 22183 tablet_bootstrap.cc:654] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.322217 22184 raft_consensus.cc:359] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.322928 22168 heartbeater.cc:499] Master 127.9.11.126:34477 was elected leader, sending a full tablet report...
I20260709 12:15:23.323108 22184 raft_consensus.cc:385] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.323764 22184 raft_consensus.cc:740] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.324630 22184 consensus_queue.cc:260] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.333281 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
W20260709 12:15:23.335616 22035 leader_election.cc:343] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.336143 22035 leader_election.cc:304] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8; no voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.336613 22184 ts_tablet_manager.cc:1434] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 12:15:23.337329 22189 raft_consensus.cc:2749] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:15:23.338341 22095 tablet.cc:2406] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:15:23.348212 22185 ts_tablet_manager.cc:1434] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.114s	user 0.026s	sys 0.030s
I20260709 12:15:23.349563 22184 tablet_bootstrap.cc:492] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.349948 22185 tablet_bootstrap.cc:492] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.353361 22183 tablet_bootstrap.cc:492] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.354061 22183 ts_tablet_manager.cc:1403] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.015s
I20260709 12:15:23.357925 22183 raft_consensus.cc:359] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.358417 22184 tablet_bootstrap.cc:654] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.358800 22183 raft_consensus.cc:385] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.359161 22183 raft_consensus.cc:740] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.360282 22183 consensus_queue.cc:260] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.370534 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.371510 22183 ts_tablet_manager.cc:1434] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.006s
W20260709 12:15:23.372593 21960 leader_election.cc:343] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.372965 22183 tablet_bootstrap.cc:492] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.375788 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
W20260709 12:15:23.377488 21960 leader_election.cc:343] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.377990 21960 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:23.378687 22185 tablet_bootstrap.cc:654] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.379052 22190 raft_consensus.cc:2749] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.385905 22185 tablet_bootstrap.cc:492] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.386554 22185 ts_tablet_manager.cc:1403] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.007s
I20260709 12:15:23.389406 22185 raft_consensus.cc:359] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.390043 22185 raft_consensus.cc:385] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.390249 22183 tablet_bootstrap.cc:654] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.390385 22185 raft_consensus.cc:740] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.391583 22185 consensus_queue.cc:260] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.393484 22184 tablet_bootstrap.cc:492] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.393826 22185 ts_tablet_manager.cc:1434] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:15:23.394165 22184 ts_tablet_manager.cc:1403] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.045s	user 0.005s	sys 0.030s
I20260709 12:15:23.394964 22185 tablet_bootstrap.cc:492] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.399821 22183 tablet_bootstrap.cc:492] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.400461 22183 ts_tablet_manager.cc:1403] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.008s
I20260709 12:15:23.401070 22184 raft_consensus.cc:359] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.401958 22184 raft_consensus.cc:385] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.402276 22184 raft_consensus.cc:740] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.404179 22195 raft_consensus.cc:493] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.404116 22183 raft_consensus.cc:359] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.405114 22183 raft_consensus.cc:385] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.404891 22195 raft_consensus.cc:515] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.405452 22183 raft_consensus.cc:740] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.405531 22185 tablet_bootstrap.cc:654] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.406548 22183 consensus_queue.cc:260] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.411731 22183 ts_tablet_manager.cc:1434] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:15:23.414858 22183 tablet_bootstrap.cc:492] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.418593 22195 leader_election.cc:290] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.403301 22184 consensus_queue.cc:260] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.425601 22189 raft_consensus.cc:493] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.426293 22189 raft_consensus.cc:515] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.428730 22183 tablet_bootstrap.cc:654] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.430105 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.430859 22189 leader_election.cc:290] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.431166 21994 raft_consensus.cc:2468] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.433172 22032 leader_election.cc:304] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.430322 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.435184 22143 raft_consensus.cc:2468] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.437060 22189 raft_consensus.cc:2804] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.437572 22189 raft_consensus.cc:493] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.437937 22189 raft_consensus.cc:3060] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.441594 22184 ts_tablet_manager.cc:1434] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.047s	user 0.003s	sys 0.007s
I20260709 12:15:23.456759 22184 tablet_bootstrap.cc:492] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.448802 22189 raft_consensus.cc:515] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.455116 22185 tablet_bootstrap.cc:492] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.459540 22185 ts_tablet_manager.cc:1403] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.065s	user 0.028s	sys 0.011s
I20260709 12:15:23.463490 22185 raft_consensus.cc:359] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.464290 22185 raft_consensus.cc:385] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.464605 22185 raft_consensus.cc:740] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.464901 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.465420 22185 consensus_queue.cc:260] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
W20260709 12:15:23.467463 22109 leader_election.cc:343] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.469310 22189 leader_election.cc:290] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.470191 22189 raft_consensus.cc:493] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.470304 22190 raft_consensus.cc:493] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.471119 22190 raft_consensus.cc:515] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.473903 22185 ts_tablet_manager.cc:1434] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:15:23.474323 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.475606 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:23.476609 22143 raft_consensus.cc:3060] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.477509 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.478353 22142 raft_consensus.cc:2468] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 0.
I20260709 12:15:23.479954 21960 leader_election.cc:304] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.480453 22184 tablet_bootstrap.cc:654] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.482868 22185 tablet_bootstrap.cc:492] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.483261 21994 raft_consensus.cc:3060] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.489411 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.490614 21993 raft_consensus.cc:2468] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.492326 22185 tablet_bootstrap.cc:654] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.492944 22106 leader_election.cc:304] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8
I20260709 12:15:23.494388 22195 raft_consensus.cc:2804] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.494812 22195 raft_consensus.cc:493] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.495223 22195 raft_consensus.cc:3060] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.495782 22183 tablet_bootstrap.cc:492] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.496503 22183 ts_tablet_manager.cc:1403] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.082s	user 0.041s	sys 0.029s
I20260709 12:15:23.499706 22190 leader_election.cc:290] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.500367 22190 raft_consensus.cc:2804] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.500181 22183 raft_consensus.cc:359] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.501217 22183 raft_consensus.cc:385] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.501605 22183 raft_consensus.cc:740] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.502521 22183 consensus_queue.cc:260] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.505362 22183 ts_tablet_manager.cc:1434] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:15:23.507380 22183 tablet_bootstrap.cc:492] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.510720 22185 tablet_bootstrap.cc:492] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.511425 22185 ts_tablet_manager.cc:1403] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.003s
I20260709 12:15:23.500802 22190 raft_consensus.cc:493] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.515566 22190 raft_consensus.cc:3060] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.470705 22189 raft_consensus.cc:515] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.514858 22183 tablet_bootstrap.cc:654] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.519352 22185 raft_consensus.cc:359] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.520406 22185 raft_consensus.cc:385] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.521577 22185 raft_consensus.cc:740] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.521874 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673096df60ed44bc8b2a9856e6e1b884" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.522670 21993 raft_consensus.cc:2468] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.522621 22185 consensus_queue.cc:260] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.524531 22032 leader_election.cc:304] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.526034 22208 raft_consensus.cc:2804] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.526535 22208 raft_consensus.cc:493] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.526976 22208 raft_consensus.cc:3060] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.537854 22195 raft_consensus.cc:515] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.540337 22185 ts_tablet_manager.cc:1434] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 12:15:23.541731 22185 tablet_bootstrap.cc:492] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.542968 22183 tablet_bootstrap.cc:492] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.543645 22183 ts_tablet_manager.cc:1403] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.009s
I20260709 12:15:23.550072 22185 tablet_bootstrap.cc:654] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.542593 21994 raft_consensus.cc:2468] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:23.546355 22208 raft_consensus.cc:515] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.555953 22032 leader_election.cc:304] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.558343 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673096df60ed44bc8b2a9856e6e1b884" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:23.560120 22208 leader_election.cc:290] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.561058 22208 raft_consensus.cc:2804] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.564030 22183 raft_consensus.cc:359] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.564883 22183 raft_consensus.cc:385] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.565003 22184 tablet_bootstrap.cc:492] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.565356 22183 raft_consensus.cc:740] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.565915 22184 ts_tablet_manager.cc:1403] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.025s
W20260709 12:15:23.564949 22035 leader_election.cc:343] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.569547 22189 leader_election.cc:290] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.570346 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:23.568272 22195 leader_election.cc:290] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.559338 22143 raft_consensus.cc:2468] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:23.572038 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673096df60ed44bc8b2a9856e6e1b884" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.572172 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673096df60ed44bc8b2a9856e6e1b884" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.572834 21994 raft_consensus.cc:3060] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.572901 22208 raft_consensus.cc:697] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
W20260709 12:15:23.573145 21960 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.67:33391, user_credentials={real_user=slave}} blocked reactor thread for 93336.8us
W20260709 12:15:23.574429 22035 leader_election.cc:343] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.574059 22208 consensus_queue.cc:237] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.575203 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.576377 22184 raft_consensus.cc:359] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.577129 22184 raft_consensus.cc:385] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.568699 22190 raft_consensus.cc:515] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.566344 22183 consensus_queue.cc:260] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.577430 22184 raft_consensus.cc:740] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.578254 22069 raft_consensus.cc:3060] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.578860 22184 consensus_queue.cc:260] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.580631 22185 tablet_bootstrap.cc:492] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.580837 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:23.581408 22185 ts_tablet_manager.cc:1403] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.010s
I20260709 12:15:23.581729 22142 raft_consensus.cc:2393] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e2a89f53d77e4029be05d7a9d2172586 in current term 1: Already voted for candidate 8600e33421784ecb83a556ef90148dd3 in this term.
I20260709 12:15:23.582531 22184 ts_tablet_manager.cc:1434] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.004s
I20260709 12:15:23.579545 22190 leader_election.cc:290] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Requested vote from peers 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.583312 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.584218 21993 raft_consensus.cc:2393] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8600e33421784ecb83a556ef90148dd3 in current term 1: Already voted for candidate e2a89f53d77e4029be05d7a9d2172586 in this term.
I20260709 12:15:23.585246 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:23.584983 22185 raft_consensus.cc:359] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.585892 22185 raft_consensus.cc:385] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.587294 21960 leader_election.cc:304] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:23.586215 22185 raft_consensus.cc:740] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.588239 22190 raft_consensus.cc:2749] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.589105 22185 consensus_queue.cc:260] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.590535 22069 raft_consensus.cc:2468] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:23.591702 22185 ts_tablet_manager.cc:1434] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:15:23.592272 22189 raft_consensus.cc:493] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.592929 22185 tablet_bootstrap.cc:492] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.592831 22189 raft_consensus.cc:515] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.596428 22189 leader_election.cc:290] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.596513 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
W20260709 12:15:23.598140 22035 leader_election.cc:343] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.597589 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
W20260709 12:15:23.600036 22032 leader_election.cc:343] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.600175 22185 tablet_bootstrap.cc:654] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.600675 22184 tablet_bootstrap.cc:492] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.600612 22032 leader_election.cc:304] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8; no voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.602202 22215 raft_consensus.cc:2749] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.605319 22183 ts_tablet_manager.cc:1434] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.000s
I20260709 12:15:23.613149 22183 tablet_bootstrap.cc:492] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.616079 22184 tablet_bootstrap.cc:654] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.617305 21994 raft_consensus.cc:2468] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:23.619089 22032 leader_election.cc:304] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:23.619992 22208 raft_consensus.cc:2804] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.620674 22185 tablet_bootstrap.cc:492] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.620709 22208 raft_consensus.cc:697] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:23.621452 22185 ts_tablet_manager.cc:1403] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 12:15:23.621930 22208 consensus_queue.cc:237] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.625020 22185 raft_consensus.cc:359] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.626053 22185 raft_consensus.cc:385] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.624887 21892 catalog_manager.cc:5697] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:23.632400 22184 tablet_bootstrap.cc:492] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.626442 22185 raft_consensus.cc:740] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.633170 22184 ts_tablet_manager.cc:1403] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.006s
I20260709 12:15:23.635717 22109 leader_election.cc:304] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.637454 22195 raft_consensus.cc:2804] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.635825 22185 consensus_queue.cc:260] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.638535 22184 raft_consensus.cc:359] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.639072 22213 raft_consensus.cc:493] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.640878 22213 raft_consensus.cc:515] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.644841 22213 leader_election.cc:290] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.645051 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.646008 22184 raft_consensus.cc:385] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.646113 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.646529 22184 raft_consensus.cc:740] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.646970 22069 raft_consensus.cc:2468] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.647819 22184 consensus_queue.cc:260] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.648634 22109 leader_election.cc:304] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:23.649243 21994 raft_consensus.cc:2468] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.649715 22213 raft_consensus.cc:2804] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.650192 22213 raft_consensus.cc:493] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.650539 22213 raft_consensus.cc:3060] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.651760 22184 ts_tablet_manager.cc:1434] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:15:23.639997 22183 tablet_bootstrap.cc:654] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.655190 22184 tablet_bootstrap.cc:492] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.664008 22213 raft_consensus.cc:515] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.674070 22184 tablet_bootstrap.cc:654] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.675076 22213 leader_election.cc:290] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.676537 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:23.677477 22069 raft_consensus.cc:3060] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.677838 22183 tablet_bootstrap.cc:492] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.678448 22183 ts_tablet_manager.cc:1403] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.005s
I20260709 12:15:23.676537 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.678843 22195 raft_consensus.cc:697] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:23.679127 21994 raft_consensus.cc:3060] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.681591 22185 ts_tablet_manager.cc:1434] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.060s	user 0.024s	sys 0.015s
I20260709 12:15:23.682878 22185 tablet_bootstrap.cc:492] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.683113 22183 raft_consensus.cc:359] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.683936 22183 raft_consensus.cc:385] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.684245 22183 raft_consensus.cc:740] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.684326 22213 raft_consensus.cc:493] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.684954 22213 raft_consensus.cc:515] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.685354 22183 consensus_queue.cc:260] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.686708 22222 raft_consensus.cc:493] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.687990 22213 leader_election.cc:290] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.688608 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673096df60ed44bc8b2a9856e6e1b884" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.689827 21993 raft_consensus.cc:2393] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8600e33421784ecb83a556ef90148dd3 in current term 1: Already voted for candidate 738c8826f8f5430d937cdbb7805bdfb8 in this term.
I20260709 12:15:23.689828 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673096df60ed44bc8b2a9856e6e1b884" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.691267 22183 ts_tablet_manager.cc:1434] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:15:23.688659 22222 raft_consensus.cc:515] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.692550 22183 tablet_bootstrap.cc:492] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.692896 22109 leader_election.cc:304] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3; no voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.694646 22222 leader_election.cc:290] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.694890 22185 tablet_bootstrap.cc:654] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.697014 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.695647 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.695641 22223 raft_consensus.cc:3060] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.698112 21993 raft_consensus.cc:2468] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.698276 22069 raft_consensus.cc:2468] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:23.697887 22068 raft_consensus.cc:2468] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.700892 22109 leader_election.cc:304] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:23.701722 22222 raft_consensus.cc:2804] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.702158 22222 raft_consensus.cc:493] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.701890 22195 consensus_queue.cc:237] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:23.702574 22222 raft_consensus.cc:3060] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.698629 21994 raft_consensus.cc:2468] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:23.706719 22109 leader_election.cc:304] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:23.707024 22184 tablet_bootstrap.cc:492] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.707710 22184 ts_tablet_manager.cc:1403] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.003s
I20260709 12:15:23.709510 22213 raft_consensus.cc:2804] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.725214 22213 raft_consensus.cc:697] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:23.726905 22184 raft_consensus.cc:359] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.727967 22184 raft_consensus.cc:385] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.728435 22184 raft_consensus.cc:740] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.727921 22213 consensus_queue.cc:237] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.729285 22184 consensus_queue.cc:260] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.739843 22222 raft_consensus.cc:515] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.742547 22222 leader_election.cc:290] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.743789 22209 raft_consensus.cc:493] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.744324 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:23.744992 22069 raft_consensus.cc:3060] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.745851 22183 tablet_bootstrap.cc:654] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.747800 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.748492 21994 raft_consensus.cc:3060] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.744385 22209 raft_consensus.cc:515] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.754029 22185 tablet_bootstrap.cc:492] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.737433 21891 catalog_manager.cc:5697] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:23.748759 22184 ts_tablet_manager.cc:1434] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 12:15:23.758378 22184 tablet_bootstrap.cc:492] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.756646 22185 ts_tablet_manager.cc:1403] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.074s	user 0.006s	sys 0.017s
I20260709 12:15:23.762447 22185 raft_consensus.cc:359] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.766978 22223 raft_consensus.cc:2749] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.772215 21994 raft_consensus.cc:2468] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:23.773337 22184 tablet_bootstrap.cc:654] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.776072 22106 leader_election.cc:304] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.777415 22213 raft_consensus.cc:2804] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.777473 22185 raft_consensus.cc:385] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.777994 22185 raft_consensus.cc:740] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.778614 22208 raft_consensus.cc:493] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.779197 22208 raft_consensus.cc:515] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.779320 22185 consensus_queue.cc:260] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.779953 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.782505 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.782747 22185 ts_tablet_manager.cc:1434] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:15:23.783141 22068 raft_consensus.cc:2468] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 0.
I20260709 12:15:23.784017 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.784487 22185 tablet_bootstrap.cc:492] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.784660 21960 leader_election.cc:304] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.785705 22209 leader_election.cc:290] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
W20260709 12:15:23.786543 22035 leader_election.cc:343] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.785705 22190 raft_consensus.cc:2804] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.787375 22190 raft_consensus.cc:493] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.787748 22190 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.791352 22213 raft_consensus.cc:697] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
W20260709 12:15:23.792384 21960 leader_election.cc:343] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.792657 22213 consensus_queue.cc:237] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.796047 22069 raft_consensus.cc:2468] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:23.796854 22208 leader_election.cc:290] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.801054 22185 tablet_bootstrap.cc:654] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.802075 22184 tablet_bootstrap.cc:492] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.802721 22184 ts_tablet_manager.cc:1403] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.025s
I20260709 12:15:23.803668 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.804399 21889 catalog_manager.cc:5697] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:23.809427 22185 tablet_bootstrap.cc:492] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.810087 22185 ts_tablet_manager.cc:1403] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.000s
I20260709 12:15:23.806399 22184 raft_consensus.cc:359] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.812974 22184 raft_consensus.cc:385] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.813009 22185 raft_consensus.cc:359] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.813555 22184 raft_consensus.cc:740] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.813822 22185 raft_consensus.cc:385] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.814138 22185 raft_consensus.cc:740] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.813874 22190 raft_consensus.cc:515] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.814374 22184 consensus_queue.cc:260] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.815073 22185 consensus_queue.cc:260] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.815855 21994 raft_consensus.cc:2393] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 738c8826f8f5430d937cdbb7805bdfb8 in current term 1: Already voted for candidate e2a89f53d77e4029be05d7a9d2172586 in this term.
I20260709 12:15:23.816919 22184 ts_tablet_manager.cc:1434] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:15:23.817613 22185 ts_tablet_manager.cc:1434] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:15:23.818041 22184 tablet_bootstrap.cc:492] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:23.818661 22185 tablet_bootstrap.cc:492] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:23.820118 22190 leader_election.cc:290] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.824162 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:23.825151 22185 tablet_bootstrap.cc:654] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.825151 22184 tablet_bootstrap.cc:654] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:23.825940 21960 leader_election.cc:343] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.827306 22183 tablet_bootstrap.cc:492] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.827983 22183 ts_tablet_manager.cc:1403] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.136s	user 0.015s	sys 0.006s
I20260709 12:15:23.827381 22032 leader_election.cc:304] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8; no voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.829041 22213 raft_consensus.cc:493] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.829944 22208 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.829804 22213 raft_consensus.cc:515] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.832008 22183 raft_consensus.cc:359] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.832808 22183 raft_consensus.cc:385] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.832255 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:23.834249 22213 leader_election.cc:290] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.835526 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
W20260709 12:15:23.837144 22109 leader_election.cc:343] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.833832 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.833218 22183 raft_consensus.cc:740] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
W20260709 12:15:23.842885 22106 leader_election.cc:343] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:23.843436 22106 leader_election.cc:304] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3; no voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.843078 22183 consensus_queue.cc:260] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.844259 22213 raft_consensus.cc:2749] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.845517 22183 ts_tablet_manager.cc:1434] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:15:23.847204 22215 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.847733 22215 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.850098 22215 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.850840 22213 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.851428 22213 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.846740 22183 tablet_bootstrap.cc:492] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:23.852494 21892 catalog_manager.cc:5697] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:23.853814 22213 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.855108 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:23.854765 21892 catalog_manager.cc:5697] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:23.855921 22068 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.856936 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.857653 21993 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:23.857753 22109 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:23.855198 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.856985 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.858891 22142 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.860150 22213 raft_consensus.cc:2804] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.860805 22035 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:23.861989 22215 raft_consensus.cc:2804] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.862491 22215 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.859231 21994 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.863125 22215 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.863137 22213 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.867658 22213 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.868146 22208 raft_consensus.cc:2749] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.880101 22183 tablet_bootstrap.cc:654] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:23.882011 22069 raft_consensus.cc:2468] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 1.
I20260709 12:15:23.887725 22215 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.891247 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.892150 21994 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.894049 22215 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.895618 22184 tablet_bootstrap.cc:492] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:23.895453 21960 leader_election.cc:304] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:23.896201 22184 ts_tablet_manager.cc:1403] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.012s
I20260709 12:15:23.896127 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:23.896876 22209 raft_consensus.cc:2804] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.898118 22209 raft_consensus.cc:697] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:23.899267 22185 tablet_bootstrap.cc:492] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:23.899811 22185 ts_tablet_manager.cc:1403] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.081s	user 0.022s	sys 0.026s
I20260709 12:15:23.899848 22209 consensus_queue.cc:237] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.901371 22213 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.902604 22142 raft_consensus.cc:2393] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 738c8826f8f5430d937cdbb7805bdfb8 in current term 1: Already voted for candidate 8600e33421784ecb83a556ef90148dd3 in this term.
I20260709 12:15:23.903767 22213 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:23.904985 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.905849 22185 raft_consensus.cc:359] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.906554 22185 raft_consensus.cc:385] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.907060 22185 raft_consensus.cc:740] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.907827 22185 consensus_queue.cc:260] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.906456 21994 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:23.909826 22184 raft_consensus.cc:359] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.910223 22032 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:23.910593 22184 raft_consensus.cc:385] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.911011 22184 raft_consensus.cc:740] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.911679 22215 raft_consensus.cc:2804] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.912294 22215 raft_consensus.cc:697] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:23.911808 22184 consensus_queue.cc:260] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.913586 22185 ts_tablet_manager.cc:1434] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:15:23.913233 22215 consensus_queue.cc:237] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.915488 22184 ts_tablet_manager.cc:1434] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:15:23.909106 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:23.918110 22109 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3; no voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:23.919088 21889 catalog_manager.cc:5697] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:23.921361 22183 tablet_bootstrap.cc:492] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:23.921969 22183 ts_tablet_manager.cc:1403] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.000s
I20260709 12:15:23.922377 22213 raft_consensus.cc:2749] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:23.925241 22183 raft_consensus.cc:359] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.925810 22183 raft_consensus.cc:385] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:23.926095 22183 raft_consensus.cc:740] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:23.927300 22183 consensus_queue.cc:260] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.929701 22183 ts_tablet_manager.cc:1434] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:15:23.933694 21892 catalog_manager.cc:5697] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:23.958217 22215 raft_consensus.cc:493] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.958818 22215 raft_consensus.cc:515] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.960843 22215 leader_election.cc:290] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.961941 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:23.962072 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:23.962785 21994 raft_consensus.cc:2468] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.962878 22142 raft_consensus.cc:2468] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:23.964102 22032 leader_election.cc:304] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.964918 22215 raft_consensus.cc:2804] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:23.965322 22215 raft_consensus.cc:493] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.965620 22215 raft_consensus.cc:3060] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.973668 22215 raft_consensus.cc:515] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.975709 22215 leader_election.cc:290] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:23.976545 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:23.976850 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:23.977245 21994 raft_consensus.cc:3060] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.977478 22142 raft_consensus.cc:3060] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:23.983808 21994 raft_consensus.cc:2468] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:23.983809 22142 raft_consensus.cc:2468] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:23.985159 22032 leader_election.cc:304] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:23.985963 22215 raft_consensus.cc:2804] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:23.986486 22215 raft_consensus.cc:697] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:23.987396 22215 consensus_queue.cc:237] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:23.995890 21892 catalog_manager.cc:5697] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.003321 22215 consensus_queue.cc:1048] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.011597 22215 consensus_queue.cc:1048] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:24.037585 22189 consensus_queue.cc:1048] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.042851 22208 consensus_queue.cc:1048] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.075441 22213 raft_consensus.cc:493] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.076025 22213 raft_consensus.cc:515] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.078190 22213 leader_election.cc:290] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:24.079021 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:24.079434 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:24.080300 22069 raft_consensus.cc:2393] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8600e33421784ecb83a556ef90148dd3 in current term 1: Already voted for candidate e2a89f53d77e4029be05d7a9d2172586 in this term.
I20260709 12:15:24.081307 22109 leader_election.cc:304] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3; no voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:24.082088 22213 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:24.088773 22213 raft_consensus.cc:2749] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:24.138309 22213 raft_consensus.cc:493] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.139030 22213 raft_consensus.cc:515] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.141314 22213 leader_election.cc:290] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:24.142364 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:24.142529 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:24.143208 21993 raft_consensus.cc:2468] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:24.143343 22069 raft_consensus.cc:2468] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:24.144555 22106 leader_election.cc:304] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.145577 22223 raft_consensus.cc:2804] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:24.145944 22223 raft_consensus.cc:493] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.146312 22223 raft_consensus.cc:3060] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:24.148993 22213 consensus_queue.cc:1048] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:24.155123 22223 raft_consensus.cc:515] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.157873 22223 leader_election.cc:290] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:24.158268 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:24.159050 21993 raft_consensus.cc:3060] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:24.162359 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:24.163113 22069 raft_consensus.cc:3060] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:24.163558 22223 consensus_queue.cc:1048] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:24.169576 21993 raft_consensus.cc:2468] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:24.173372 22106 leader_election.cc:304] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.175333 22195 raft_consensus.cc:2804] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:24.176090 22195 raft_consensus.cc:697] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:24.177492 22195 consensus_queue.cc:237] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.178660 22069 raft_consensus.cc:2468] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:24.181927 22223 consensus_queue.cc:1048] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.191931 21889 catalog_manager.cc:5697] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.196743 22195 consensus_queue.cc:1048] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.229796 22143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:24.230491 22143 raft_consensus.cc:606] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:24.236080 22143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:24.236608 22143 raft_consensus.cc:606] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:24.251199 22069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36940
I20260709 12:15:24.251777 22069 raft_consensus.cc:606] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:24.257627 22069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36940
I20260709 12:15:24.258253 22069 raft_consensus.cc:606] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:24.259218 22195 consensus_queue.cc:1048] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.264672 22143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:24.265446 22143 raft_consensus.cc:606] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS 738c8826f8f5430d937cdbb7805bdfb8
I20260709 12:15:24.267877 22223 raft_consensus.cc:993] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: : Instructing follower 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:24.268368 22213 raft_consensus.cc:1081] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:24.275125 22195 consensus_queue.cc:1048] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:15:24.281023 22069 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36928
I20260709 12:15:24.281920 22069 raft_consensus.cc:493] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.289639 22069 raft_consensus.cc:3060] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.300071 22069 raft_consensus.cc:515] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.304692 22069 leader_election.cc:290] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:24.305584 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:24.306416 21994 raft_consensus.cc:3060] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.307214 22209 consensus_queue.cc:1048] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.309080 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f55e2d5913649e6a00e298a453cc96a" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.310897 22143 raft_consensus.cc:3055] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.311275 22143 raft_consensus.cc:740] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:24.312192 22143 consensus_queue.cc:260] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.313768 22143 raft_consensus.cc:3060] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.313807 21992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f84307224ba4c489d4120bc91039178"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
mode: GRACEFUL
new_leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36418
I20260709 12:15:24.314550 21992 raft_consensus.cc:606] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Received request to transfer leadership to TS 738c8826f8f5430d937cdbb7805bdfb8
I20260709 12:15:24.318141 21994 raft_consensus.cc:2468] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:24.324581 22032 leader_election.cc:304] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.325146 22142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:24.325687 22208 raft_consensus.cc:2804] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.325974 22142 raft_consensus.cc:606] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS 738c8826f8f5430d937cdbb7805bdfb8
I20260709 12:15:24.326360 22208 raft_consensus.cc:697] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:24.327427 22208 consensus_queue.cc:237] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.328605 22190 raft_consensus.cc:993] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: : Instructing follower 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:24.329562 22209 raft_consensus.cc:1081] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Signalling peer 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:24.332501 22068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f84307224ba4c489d4120bc91039178"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36912
I20260709 12:15:24.334227 22068 raft_consensus.cc:493] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.334667 22068 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.337167 22069 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
mode: GRACEFUL
new_leader_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:36940
I20260709 12:15:24.338043 22069 raft_consensus.cc:606] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Received request to transfer leadership to TS 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:24.345635 22143 raft_consensus.cc:2468] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:24.345674 22068 raft_consensus.cc:515] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.353281 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:24.355499 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.354828 21891 catalog_manager.cc:5697] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.356228 22142 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.355948 22209 consensus_queue.cc:1048] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.352069 22068 leader_election.cc:290] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:24.358706 21994 raft_consensus.cc:3055] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.359136 21994 raft_consensus.cc:740] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:24.360112 21994 consensus_queue.cc:260] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.361167 21994 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.365638 22142 raft_consensus.cc:2468] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:24.366873 22141 raft_consensus.cc:1240] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Rejecting Update request from peer e2a89f53d77e4029be05d7a9d2172586 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:15:24.367347 22035 leader_election.cc:304] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:24.368613 22208 raft_consensus.cc:2804] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.369309 22208 raft_consensus.cc:697] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:24.370411 22208 consensus_queue.cc:237] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.372176 21994 raft_consensus.cc:2468] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:24.379174 21889 catalog_manager.cc:5697] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 1 to 2, leader changed from e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65) to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.433140 22208 consensus_queue.cc:1048] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:24.445555 22208 consensus_queue.cc:1048] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.455094 22215 raft_consensus.cc:993] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.455687 22215 raft_consensus.cc:1081] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.457638 21994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36396
I20260709 12:15:24.457551 22215 consensus_queue.cc:1048] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:24.458321 21994 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.458771 21994 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.465325 22208 raft_consensus.cc:993] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: : Instructing follower 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:24.465946 22208 raft_consensus.cc:1081] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Signalling peer 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:24.467515 21994 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.468681 22142 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:51388
I20260709 12:15:24.472057 21994 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:24.474117 22142 raft_consensus.cc:493] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.474129 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.474604 22142 raft_consensus.cc:3060] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.474789 22189 consensus_queue.cc:1048] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.474968 22141 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.480106 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:24.480763 22068 raft_consensus.cc:3055] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.481110 22068 raft_consensus.cc:740] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:24.482056 22068 consensus_queue.cc:260] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.484005 22068 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.489619 22142 raft_consensus.cc:515] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.497359 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:24.497556 22142 leader_election.cc:290] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:24.498144 21992 raft_consensus.cc:3060] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.498286 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "335b3b26e87f404fa95b6d2a3c85f605" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:24.499074 22069 raft_consensus.cc:3055] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.499408 22069 raft_consensus.cc:740] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:24.500325 22069 consensus_queue.cc:260] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.501827 22069 raft_consensus.cc:3060] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.505439 22141 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.506875 21960 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.507643 21992 raft_consensus.cc:2468] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 2.
I20260709 12:15:24.507875 22209 raft_consensus.cc:2804] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.508523 22209 raft_consensus.cc:697] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:24.509358 22106 leader_election.cc:304] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.509539 22209 consensus_queue.cc:237] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.511240 22223 raft_consensus.cc:2804] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.511794 22223 raft_consensus.cc:697] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:24.512115 22068 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.512769 22223 consensus_queue.cc:237] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.513341 22069 raft_consensus.cc:2468] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 2.
I20260709 12:15:24.516803 22215 raft_consensus.cc:993] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.517341 22208 raft_consensus.cc:1081] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.519455 21994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36396
I20260709 12:15:24.520016 21994 raft_consensus.cc:493] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.520375 21994 raft_consensus.cc:3060] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.523505 21889 catalog_manager.cc:5697] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 1 to 2, leader changed from 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66) to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:24.528601 21994 raft_consensus.cc:515] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.530968 21994 leader_election.cc:290] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:24.532015 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:24.532344 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fcbeb90ec3641b88309d64a33a7fe90" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.532856 22069 raft_consensus.cc:3055] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.533046 22141 raft_consensus.cc:3060] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.533288 22069 raft_consensus.cc:740] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:24.534673 22069 consensus_queue.cc:260] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.534767 21891 catalog_manager.cc:5697] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.536443 22069 raft_consensus.cc:3060] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.542335 22141 raft_consensus.cc:2468] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.544296 21960 leader_election.cc:304] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.545209 22209 raft_consensus.cc:2804] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.545747 22209 raft_consensus.cc:697] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:24.546747 22209 consensus_queue.cc:237] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.551003 22069 raft_consensus.cc:2468] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.554040 21889 catalog_manager.cc:5697] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.613327 22223 consensus_queue.cc:1048] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:24.620688 22222 raft_consensus.cc:993] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.621282 22222 raft_consensus.cc:1081] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.625813 21994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36408
I20260709 12:15:24.627332 21994 raft_consensus.cc:493] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.627890 21994 raft_consensus.cc:3060] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.628633 22213 consensus_queue.cc:1048] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:24.638713 21994 raft_consensus.cc:515] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:24.640260 22222 raft_consensus.cc:993] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: : Instructing follower 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:24.640887 22213 raft_consensus.cc:1081] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:24.641350 21994 leader_election.cc:290] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:24.642488 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:24.643432 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2339e13a2b06478b8e08f568b25762a1" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.643334 22068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36928
I20260709 12:15:24.644174 22069 raft_consensus.cc:3060] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.644321 22068 raft_consensus.cc:493] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.644174 22141 raft_consensus.cc:3055] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.644914 22068 raft_consensus.cc:3060] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.645037 22141 raft_consensus.cc:740] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:24.646090 22141 consensus_queue.cc:260] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:24.647583 22141 raft_consensus.cc:3060] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.652089 22068 raft_consensus.cc:515] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.653211 22069 raft_consensus.cc:2468] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.654563 22068 leader_election.cc:290] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:24.655459 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:24.655584 21960 leader_election.cc:304] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.656271 21994 raft_consensus.cc:3060] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.656613 22209 raft_consensus.cc:2804] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.657188 22209 raft_consensus.cc:697] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:24.657416 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c965fb2a6f24b2eace0008b8f83bbbb" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.658142 22142 raft_consensus.cc:3055] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.658593 22142 raft_consensus.cc:740] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:24.658185 22209 consensus_queue.cc:237] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } }
I20260709 12:15:24.659569 22142 consensus_queue.cc:260] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.661171 22142 raft_consensus.cc:3060] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.665516 21994 raft_consensus.cc:2468] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:24.665540 22141 raft_consensus.cc:2468] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.667441 22032 leader_election.cc:304] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.668424 22208 raft_consensus.cc:2804] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.668941 22208 raft_consensus.cc:697] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:24.669842 22208 consensus_queue.cc:237] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.670485 22142 raft_consensus.cc:2468] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:24.671212 21891 catalog_manager.cc:5697] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.679432 21889 catalog_manager.cc:5697] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.688833 22222 raft_consensus.cc:993] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.689561 22213 raft_consensus.cc:1081] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:24.691169 21994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36408
I20260709 12:15:24.691725 21994 raft_consensus.cc:493] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:24.692143 21994 raft_consensus.cc:3060] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.698264 21994 raft_consensus.cc:515] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.699856 21994 leader_election.cc:290] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:24.701125 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:24.701381 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cbb5d889cc3467e9a166982b4eacfe0" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:24.701884 22068 raft_consensus.cc:3060] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.702062 22142 raft_consensus.cc:3055] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:24.702482 22142 raft_consensus.cc:740] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:24.703338 22142 consensus_queue.cc:260] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.704640 22142 raft_consensus.cc:3060] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:24.708871 22068 raft_consensus.cc:2468] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.710345 21960 leader_election.cc:304] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:24.711257 22209 raft_consensus.cc:2804] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:24.711479 22142 raft_consensus.cc:2468] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:24.711740 22209 raft_consensus.cc:697] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:24.712857 22209 consensus_queue.cc:237] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:24.720870 21889 catalog_manager.cc:5697] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:24.805608 22142 raft_consensus.cc:1275] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:24.807180 22208 consensus_queue.cc:1048] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:24.816272 21994 raft_consensus.cc:1275] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:24.818647 22215 consensus_queue.cc:1048] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:24.919925 22068 raft_consensus.cc:1275] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:24.921554 22209 consensus_queue.cc:1048] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:24.931762 22142 raft_consensus.cc:1275] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:24.934285 22269 consensus_queue.cc:1048] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:24.950592 21994 raft_consensus.cc:1275] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:24.951982 22215 consensus_queue.cc:1048] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:24.960153 22142 raft_consensus.cc:1275] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:24.962440 22215 consensus_queue.cc:1048] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:25.019187 22068 raft_consensus.cc:1275] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.020670 22213 consensus_queue.cc:1048] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.029598 21994 raft_consensus.cc:1275] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.031299 22222 consensus_queue.cc:1048] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.080966 22068 raft_consensus.cc:1275] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.082530 22270 consensus_queue.cc:1048] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.092967 22142 raft_consensus.cc:1275] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.095805 22262 consensus_queue.cc:1048] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.107125 22068 raft_consensus.cc:1275] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.108491 22262 consensus_queue.cc:1048] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.119807 22142 raft_consensus.cc:1275] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.121340 22262 consensus_queue.cc:1048] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.129173 22068 raft_consensus.cc:1275] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.130554 22262 consensus_queue.cc:1048] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:25.138350 21994 raft_consensus.cc:1275] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.138839 22142 raft_consensus.cc:1275] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.140043 22189 consensus_queue.cc:1048] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:25.141505 22262 consensus_queue.cc:1048] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:25.150346 22141 raft_consensus.cc:1275] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:25.152632 22189 consensus_queue.cc:1048] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:27.381024 21891 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38986:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:15:27.385371 21891 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:15:27.457777 21982 tablet_service.cc:1511] Processing CreateTablet for tablet f5da0d2a09f547a6a9457178dcd02a85 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:27.459331 22058 tablet_service.cc:1511] Processing CreateTablet for tablet f5da0d2a09f547a6a9457178dcd02a85 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:27.461479 22127 tablet_service.cc:1511] Processing CreateTablet for tablet f5da0d2a09f547a6a9457178dcd02a85 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:15:27.462333 22058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5da0d2a09f547a6a9457178dcd02a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.462916 21982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5da0d2a09f547a6a9457178dcd02a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.464562 21977 tablet_service.cc:1511] Processing CreateTablet for tablet d74c6d0fa6e943c58de2cc7643a98d93 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:27.464915 22056 tablet_service.cc:1511] Processing CreateTablet for tablet d74c6d0fa6e943c58de2cc7643a98d93 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:27.466447 21977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d74c6d0fa6e943c58de2cc7643a98d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.466794 22056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d74c6d0fa6e943c58de2cc7643a98d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.470757 22127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5da0d2a09f547a6a9457178dcd02a85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.475194 22126 tablet_service.cc:1511] Processing CreateTablet for tablet d74c6d0fa6e943c58de2cc7643a98d93 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:15:27.476984 22126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d74c6d0fa6e943c58de2cc7643a98d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.501437 22055 tablet_service.cc:1511] Processing CreateTablet for tablet c98fca4dbfcf412499ce812e14ca0a4e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:27.503530 22055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98fca4dbfcf412499ce812e14ca0a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.503968 22130 tablet_service.cc:1511] Processing CreateTablet for tablet c98fca4dbfcf412499ce812e14ca0a4e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:27.505712 22130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98fca4dbfcf412499ce812e14ca0a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.508602 21978 tablet_service.cc:1511] Processing CreateTablet for tablet c98fca4dbfcf412499ce812e14ca0a4e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:15:27.509186 22059 tablet_service.cc:1511] Processing CreateTablet for tablet 12363c88ac98469589a4893c96df2d5e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:27.510253 21978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c98fca4dbfcf412499ce812e14ca0a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.510668 22059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12363c88ac98469589a4893c96df2d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.512377 21983 tablet_service.cc:1511] Processing CreateTablet for tablet 12363c88ac98469589a4893c96df2d5e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:27.514009 21983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12363c88ac98469589a4893c96df2d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.520699 22125 tablet_service.cc:1511] Processing CreateTablet for tablet 12363c88ac98469589a4893c96df2d5e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:15:27.529783 21980 tablet_service.cc:1511] Processing CreateTablet for tablet 267ef64c6085422e98bc0add84f3442f (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:27.532660 21980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267ef64c6085422e98bc0add84f3442f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.522547 22125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12363c88ac98469589a4893c96df2d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.576758 22290 tablet_bootstrap.cc:492] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:27.582293 22289 tablet_bootstrap.cc:492] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:27.590226 22290 tablet_bootstrap.cc:654] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.599488 22056 tablet_service.cc:1511] Processing CreateTablet for tablet 267ef64c6085422e98bc0add84f3442f (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:27.603380 22058 tablet_service.cc:1511] Processing CreateTablet for tablet e8de6bdba9564589947ac9874077e391 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:27.608035 22058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8de6bdba9564589947ac9874077e391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.607853 21981 tablet_service.cc:1511] Processing CreateTablet for tablet a53af6bb389648d6b3f53d229d48e9b7 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:27.609588 21981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a53af6bb389648d6b3f53d229d48e9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.609443 22129 tablet_service.cc:1511] Processing CreateTablet for tablet e8de6bdba9564589947ac9874077e391 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:27.604022 21982 tablet_service.cc:1511] Processing CreateTablet for tablet e8de6bdba9564589947ac9874077e391 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:15:27.611043 22129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8de6bdba9564589947ac9874077e391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.612838 22057 tablet_service.cc:1511] Processing CreateTablet for tablet a53af6bb389648d6b3f53d229d48e9b7 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:27.614324 22057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a53af6bb389648d6b3f53d229d48e9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.598855 22128 tablet_service.cc:1511] Processing CreateTablet for tablet 267ef64c6085422e98bc0add84f3442f (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:15:27.616106 22128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267ef64c6085422e98bc0add84f3442f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.627719 21982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8de6bdba9564589947ac9874077e391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.631810 22056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 267ef64c6085422e98bc0add84f3442f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.634210 22132 tablet_service.cc:1511] Processing CreateTablet for tablet a53af6bb389648d6b3f53d229d48e9b7 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:15:27.635838 22132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a53af6bb389648d6b3f53d229d48e9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.640736 22289 tablet_bootstrap.cc:654] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.653586 21984 tablet_service.cc:1511] Processing CreateTablet for tablet 5583597bdfae45d7bcadd9d354e35f3e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:27.657074 21984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5583597bdfae45d7bcadd9d354e35f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.687841 22290 tablet_bootstrap.cc:492] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:27.688689 22290 ts_tablet_manager.cc:1403] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.006s
I20260709 12:15:27.703089 22293 tablet_bootstrap.cc:492] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:27.694240 22290 raft_consensus.cc:359] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.711251 21981 tablet_service.cc:1511] Processing CreateTablet for tablet ea14aee8ecd84228ab92126aa2243131 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:27.711356 22290 raft_consensus.cc:385] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.714453 22290 raft_consensus.cc:740] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.711247 22057 tablet_service.cc:1511] Processing CreateTablet for tablet 5583597bdfae45d7bcadd9d354e35f3e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:27.720408 22057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5583597bdfae45d7bcadd9d354e35f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.713510 21981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea14aee8ecd84228ab92126aa2243131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.727583 22293 tablet_bootstrap.cc:654] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.712545 22132 tablet_service.cc:1511] Processing CreateTablet for tablet 5583597bdfae45d7bcadd9d354e35f3e (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:15:27.742235 22132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5583597bdfae45d7bcadd9d354e35f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.744024 22056 tablet_service.cc:1511] Processing CreateTablet for tablet ea14aee8ecd84228ab92126aa2243131 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:27.745486 22056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea14aee8ecd84228ab92126aa2243131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.752864 22126 tablet_service.cc:1511] Processing CreateTablet for tablet ea14aee8ecd84228ab92126aa2243131 (DEFAULT_TABLE table=soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:15:27.754400 22126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea14aee8ecd84228ab92126aa2243131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:27.762560 22293 tablet_bootstrap.cc:492] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:27.763407 22293 ts_tablet_manager.cc:1403] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.061s	user 0.004s	sys 0.015s
I20260709 12:15:27.766435 22293 raft_consensus.cc:359] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.731971 22290 consensus_queue.cc:260] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.767096 22293 raft_consensus.cc:385] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.780464 22293 raft_consensus.cc:740] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.781315 22289 tablet_bootstrap.cc:492] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:27.782073 22289 ts_tablet_manager.cc:1403] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.200s	user 0.023s	sys 0.006s
I20260709 12:15:27.785879 22289 raft_consensus.cc:359] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.787616 22290 ts_tablet_manager.cc:1434] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.099s	user 0.011s	sys 0.000s
I20260709 12:15:27.786759 22289 raft_consensus.cc:385] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.795701 22289 raft_consensus.cc:740] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.796685 22289 consensus_queue.cc:260] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.781358 22293 consensus_queue.cc:260] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.806389 22290 tablet_bootstrap.cc:492] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:27.808624 22293 ts_tablet_manager.cc:1434] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260709 12:15:27.810235 22293 tablet_bootstrap.cc:492] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:27.808672 22289 ts_tablet_manager.cc:1434] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 12:15:27.811873 22289 tablet_bootstrap.cc:492] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:27.818586 22293 tablet_bootstrap.cc:654] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.829738 22290 tablet_bootstrap.cc:654] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.833542 22293 tablet_bootstrap.cc:492] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:27.834276 22293 ts_tablet_manager.cc:1403] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.013s
I20260709 12:15:27.838728 22289 tablet_bootstrap.cc:654] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.838860 22293 raft_consensus.cc:359] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.840014 22293 raft_consensus.cc:385] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.840379 22293 raft_consensus.cc:740] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.841279 22293 consensus_queue.cc:260] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.844597 22293 ts_tablet_manager.cc:1434] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.006s
I20260709 12:15:27.845227 22290 tablet_bootstrap.cc:492] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:27.845914 22290 ts_tablet_manager.cc:1403] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.000s
I20260709 12:15:27.846328 22293 tablet_bootstrap.cc:492] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:27.850189 22290 raft_consensus.cc:359] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.851055 22290 raft_consensus.cc:385] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.851432 22290 raft_consensus.cc:740] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.852054 22289 tablet_bootstrap.cc:492] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:27.852706 22289 ts_tablet_manager.cc:1403] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.003s
I20260709 12:15:27.852504 22290 consensus_queue.cc:260] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.855921 22290 ts_tablet_manager.cc:1434] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:15:27.856140 22293 tablet_bootstrap.cc:654] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.856397 22289 raft_consensus.cc:359] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.857278 22289 raft_consensus.cc:385] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.857692 22289 raft_consensus.cc:740] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.858604 22289 consensus_queue.cc:260] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.857286 22290 tablet_bootstrap.cc:492] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:27.861285 22289 ts_tablet_manager.cc:1434] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:15:27.862516 22289 tablet_bootstrap.cc:492] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:27.870428 22289 tablet_bootstrap.cc:654] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.897195 22290 tablet_bootstrap.cc:654] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.905864 22289 tablet_bootstrap.cc:492] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:27.906502 22293 tablet_bootstrap.cc:492] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:27.906538 22289 ts_tablet_manager.cc:1403] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.021s
I20260709 12:15:27.907277 22293 ts_tablet_manager.cc:1403] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.061s	user 0.029s	sys 0.031s
I20260709 12:15:27.909051 22290 tablet_bootstrap.cc:492] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:27.909657 22290 ts_tablet_manager.cc:1403] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.000s
I20260709 12:15:27.909999 22289 raft_consensus.cc:359] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.910666 22289 raft_consensus.cc:385] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.910935 22289 raft_consensus.cc:740] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.910913 22293 raft_consensus.cc:359] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.911885 22293 raft_consensus.cc:385] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.912241 22293 raft_consensus.cc:740] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.911999 22289 consensus_queue.cc:260] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.913101 22293 consensus_queue.cc:260] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.913196 22290 raft_consensus.cc:359] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.914142 22290 raft_consensus.cc:385] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.914511 22290 raft_consensus.cc:740] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.914665 22289 ts_tablet_manager.cc:1434] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:15:27.915553 22290 consensus_queue.cc:260] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.916090 22289 tablet_bootstrap.cc:492] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:27.916175 22293 ts_tablet_manager.cc:1434] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 12:15:27.917783 22293 tablet_bootstrap.cc:492] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:27.918556 22290 ts_tablet_manager.cc:1434] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:15:27.919854 22290 tablet_bootstrap.cc:492] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:27.925817 22293 tablet_bootstrap.cc:654] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.927585 22290 tablet_bootstrap.cc:654] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.937775 22289 tablet_bootstrap.cc:654] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.939847 22290 tablet_bootstrap.cc:492] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:27.940603 22290 ts_tablet_manager.cc:1403] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:15:27.942776 22293 tablet_bootstrap.cc:492] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:27.943572 22293 ts_tablet_manager.cc:1403] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.011s
I20260709 12:15:27.944656 22290 raft_consensus.cc:359] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.945616 22290 raft_consensus.cc:385] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.947086 22293 raft_consensus.cc:359] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.945995 22290 raft_consensus.cc:740] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.947940 22293 raft_consensus.cc:385] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.948318 22293 raft_consensus.cc:740] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.948737 22290 consensus_queue.cc:260] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.950111 22293 consensus_queue.cc:260] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.951716 22290 ts_tablet_manager.cc:1434] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:15:27.953022 22290 tablet_bootstrap.cc:492] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:27.953907 22293 ts_tablet_manager.cc:1434] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:15:27.957582 22293 tablet_bootstrap.cc:492] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:27.960155 22270 raft_consensus.cc:493] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.961189 22290 tablet_bootstrap.cc:654] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.960989 22270 raft_consensus.cc:515] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.966104 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98fca4dbfcf412499ce812e14ca0a4e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:27.966171 22293 tablet_bootstrap.cc:654] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:27.967331 22068 raft_consensus.cc:2468] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 0.
I20260709 12:15:27.970052 21960 leader_election.cc:304] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:27.971871 22296 raft_consensus.cc:2804] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:27.972436 22296 raft_consensus.cc:493] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.972878 22296 raft_consensus.cc:3060] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:27.975854 22290 tablet_bootstrap.cc:492] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:27.976832 22290 ts_tablet_manager.cc:1403] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.016s
I20260709 12:15:27.980279 22270 leader_election.cc:290] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:27.980830 22293 tablet_bootstrap.cc:492] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:27.981572 22293 ts_tablet_manager.cc:1403] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.007s
I20260709 12:15:27.980916 22290 raft_consensus.cc:359] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.982059 22290 raft_consensus.cc:385] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:27.982146 22270 raft_consensus.cc:493] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.982518 22290 raft_consensus.cc:740] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.982959 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98fca4dbfcf412499ce812e14ca0a4e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:27.982888 22270 raft_consensus.cc:515] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.983632 22290 consensus_queue.cc:260] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.985824 22293 raft_consensus.cc:359] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.986541 22290 ts_tablet_manager.cc:1434] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:15:27.986725 22293 raft_consensus.cc:385] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:15:27.986671 21960 leader_election.cc:343] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:27.987911 22290 tablet_bootstrap.cc:492] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:27.988097 22293 raft_consensus.cc:740] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:27.989125 22293 consensus_queue.cc:260] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:27.991870 22293 ts_tablet_manager.cc:1434] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:15:27.993403 22293 tablet_bootstrap.cc:492] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:27.997304 22290 tablet_bootstrap.cc:654] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.002138 22293 tablet_bootstrap.cc:654] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.013926 22291 raft_consensus.cc:493] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.014601 22291 raft_consensus.cc:515] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.018522 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.019357 21994 raft_consensus.cc:2468] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:28.019532 22296 raft_consensus.cc:515] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.021531 22106 leader_election.cc:304] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.023154 22277 raft_consensus.cc:2804] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.023736 22277 raft_consensus.cc:493] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.024199 22277 raft_consensus.cc:3060] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.022691 22296 leader_election.cc:290] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.031711 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.032281 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98fca4dbfcf412499ce812e14ca0a4e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:28.032835 22068 raft_consensus.cc:2468] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 0.
I20260709 12:15:28.033304 22069 raft_consensus.cc:3060] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.036123 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c98fca4dbfcf412499ce812e14ca0a4e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:28.037101 22141 raft_consensus.cc:3060] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.041118 21960 leader_election.cc:304] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.062659 22296 raft_consensus.cc:2804] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.063169 22296 raft_consensus.cc:493] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.063521 22296 raft_consensus.cc:3060] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.041097 22270 leader_election.cc:290] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.066949 22291 leader_election.cc:290] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.068054 22291 raft_consensus.cc:493] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.068738 22291 raft_consensus.cc:515] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.073489 22278 raft_consensus.cc:493] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.074199 22278 raft_consensus.cc:515] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.076545 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.076900 22278 leader_election.cc:290] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.077515 22068 raft_consensus.cc:2468] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:28.074571 22142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.081416 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.083039 22293 tablet_bootstrap.cc:492] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
W20260709 12:15:28.083420 22032 leader_election.cc:343] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.083969 22293 ts_tablet_manager.cc:1403] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.010s
I20260709 12:15:28.089766 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.090636 21994 raft_consensus.cc:2468] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:28.090669 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.091774 22068 raft_consensus.cc:2468] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:28.092141 22106 leader_election.cc:304] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.096071 22290 tablet_bootstrap.cc:492] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:28.096963 22290 ts_tablet_manager.cc:1403] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.109s	user 0.042s	sys 0.041s
I20260709 12:15:28.097048 22291 leader_election.cc:290] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.100993 22310 raft_consensus.cc:2804] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.101598 22310 raft_consensus.cc:493] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.102010 22310 raft_consensus.cc:3060] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.104193 22291 raft_consensus.cc:493] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.105057 22291 raft_consensus.cc:515] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.105963 22141 raft_consensus.cc:2468] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 1.
I20260709 12:15:28.099400 22296 raft_consensus.cc:515] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.107755 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.109673 22290 raft_consensus.cc:359] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.109194 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.110508 22290 raft_consensus.cc:385] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.110725 22067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:28.113993 22289 tablet_bootstrap.cc:492] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:28.110924 22290 raft_consensus.cc:740] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.111028 22068 raft_consensus.cc:2468] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:28.117625 22289 ts_tablet_manager.cc:1403] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.202s	user 0.057s	sys 0.062s
I20260709 12:15:28.104193 22293 raft_consensus.cc:359] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.118966 22293 raft_consensus.cc:385] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.118628 22290 consensus_queue.cc:260] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.119201 22109 leader_election.cc:304] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:28.111907 21960 leader_election.cc:304] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.119763 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:28.121434 22290 ts_tablet_manager.cc:1434] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:15:28.122823 22270 raft_consensus.cc:2804] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.116978 22067 raft_consensus.cc:3060] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.124954 22277 raft_consensus.cc:515] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.126524 22290 tablet_bootstrap.cc:492] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:28.109169 22069 raft_consensus.cc:2468] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 1.
I20260709 12:15:28.129741 22142 raft_consensus.cc:2393] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a89f53d77e4029be05d7a9d2172586 in current term 1: Already voted for candidate 8600e33421784ecb83a556ef90148dd3 in this term.
I20260709 12:15:28.132123 22312 raft_consensus.cc:2804] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.134490 22312 raft_consensus.cc:493] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.135845 22312 raft_consensus.cc:3060] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:28.136504 22106 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.11.65:46861, user_credentials={real_user=slave}} blocked reactor thread for 44486.5us
I20260709 12:15:28.143184 22277 leader_election.cc:290] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.144196 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:28.145339 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.110781 22296 leader_election.cc:290] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.138022 22290 tablet_bootstrap.cc:654] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.119320 22293 raft_consensus.cc:740] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
W20260709 12:15:28.146790 22106 leader_election.cc:343] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.131875 22289 raft_consensus.cc:359] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.145478 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:28.147526 22293 consensus_queue.cc:260] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.148110 21992 raft_consensus.cc:2393] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8600e33421784ecb83a556ef90148dd3 in current term 1: Already voted for candidate e2a89f53d77e4029be05d7a9d2172586 in this term.
I20260709 12:15:28.148674 22143 raft_consensus.cc:2468] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:28.149384 22289 raft_consensus.cc:385] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.149828 22289 raft_consensus.cc:740] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.150513 22293 ts_tablet_manager.cc:1434] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.004s
I20260709 12:15:28.150689 22289 consensus_queue.cc:260] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.140131 22291 leader_election.cc:290] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.151911 22106 leader_election.cc:304] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3; no voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.153111 22291 raft_consensus.cc:2749] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:28.154197 22035 leader_election.cc:304] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.155483 22311 raft_consensus.cc:2804] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.155834 22289 ts_tablet_manager.cc:1434] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 12:15:28.155916 22311 raft_consensus.cc:493] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.155689 22293 tablet_bootstrap.cc:492] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:28.156486 22311 raft_consensus.cc:3060] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.159596 22270 raft_consensus.cc:697] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:28.165287 22270 consensus_queue.cc:237] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.168037 22289 tablet_bootstrap.cc:492] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:28.171036 22067 raft_consensus.cc:2468] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 1.
I20260709 12:15:28.172868 21960 leader_election.cc:304] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:28.167898 22310 raft_consensus.cc:515] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.174046 22313 raft_consensus.cc:2804] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.175576 22313 raft_consensus.cc:697] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:28.176509 22313 consensus_queue.cc:237] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.178092 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:28.178912 22068 raft_consensus.cc:3060] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.176512 22310 leader_election.cc:290] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.182461 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:28.183224 21992 raft_consensus.cc:3060] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.183647 22311 raft_consensus.cc:515] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.185101 22312 raft_consensus.cc:515] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.187067 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
W20260709 12:15:28.189646 22032 leader_election.cc:343] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.190972 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:28.192261 22067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
W20260709 12:15:28.192912 22106 leader_election.cc:343] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.193347 22067 raft_consensus.cc:3060] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.191588 22312 leader_election.cc:290] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.198076 22290 tablet_bootstrap.cc:492] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:28.198773 22290 ts_tablet_manager.cc:1403] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.002s
I20260709 12:15:28.202150 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:28.202940 22143 raft_consensus.cc:3060] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.202297 21891 catalog_manager.cc:5697] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:28.203387 22290 raft_consensus.cc:359] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.204166 22290 raft_consensus.cc:385] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.204514 22290 raft_consensus.cc:740] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.205354 22290 consensus_queue.cc:260] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.209483 22289 tablet_bootstrap.cc:654] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.221868 22311 leader_election.cc:290] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.213171 22143 raft_consensus.cc:2468] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:28.217024 22290 ts_tablet_manager.cc:1434] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:15:28.223059 22067 raft_consensus.cc:2468] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:28.224529 22290 tablet_bootstrap.cc:492] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:28.202813 22293 tablet_bootstrap.cc:654] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.223069 21992 raft_consensus.cc:2468] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:28.228045 22106 leader_election.cc:304] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.217865 22068 raft_consensus.cc:2468] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:28.229020 22312 raft_consensus.cc:2804] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.229560 22312 raft_consensus.cc:697] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:28.230540 22312 consensus_queue.cc:237] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.230969 22109 leader_election.cc:304] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.233888 22310 raft_consensus.cc:2804] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.234540 22310 raft_consensus.cc:697] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:28.235705 22310 consensus_queue.cc:237] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.239837 22289 tablet_bootstrap.cc:492] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:28.242125 22289 ts_tablet_manager.cc:1403] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.074s	user 0.007s	sys 0.012s
I20260709 12:15:28.242321 22293 tablet_bootstrap.cc:492] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:28.243206 22293 ts_tablet_manager.cc:1403] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.003s
I20260709 12:15:28.245427 22290 tablet_bootstrap.cc:654] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.246891 22293 raft_consensus.cc:359] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.247815 22293 raft_consensus.cc:385] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.248162 22293 raft_consensus.cc:740] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.248075 22289 raft_consensus.cc:359] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.249071 22289 raft_consensus.cc:385] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.249432 22289 raft_consensus.cc:740] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.249284 22293 consensus_queue.cc:260] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.250615 22278 raft_consensus.cc:493] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.250619 22289 consensus_queue.cc:260] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.251224 22278 raft_consensus.cc:515] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.259080 22035 leader_election.cc:304] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.260593 22311 raft_consensus.cc:2804] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.260839 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267ef64c6085422e98bc0add84f3442f" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.261166 22278 leader_election.cc:290] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.262725 22311 raft_consensus.cc:697] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
W20260709 12:15:28.262796 22032 leader_election.cc:343] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.253216 22293 ts_tablet_manager.cc:1434] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:15:28.250259 21890 catalog_manager.cc:5697] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:28.264595 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267ef64c6085422e98bc0add84f3442f" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.260279 22290 tablet_bootstrap.cc:492] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:28.265180 22293 tablet_bootstrap.cc:492] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:28.265302 22143 raft_consensus.cc:2468] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:28.265793 22290 ts_tablet_manager.cc:1403] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.007s
I20260709 12:15:28.266667 22035 leader_election.cc:304] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.267647 22278 raft_consensus.cc:2804] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.268097 22278 raft_consensus.cc:493] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.268544 22278 raft_consensus.cc:3060] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.269182 22313 raft_consensus.cc:493] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.263998 22311 consensus_queue.cc:237] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.270219 22313 raft_consensus.cc:515] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.271755 22289 ts_tablet_manager.cc:1434] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.005s
I20260709 12:15:28.275245 22289 tablet_bootstrap.cc:492] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:28.276898 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.279003 22313 leader_election.cc:290] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.279985 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.280933 22143 raft_consensus.cc:2393] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a89f53d77e4029be05d7a9d2172586 in current term 1: Already voted for candidate 738c8826f8f5430d937cdbb7805bdfb8 in this term.
I20260709 12:15:28.280813 22290 raft_consensus.cc:359] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.283485 22290 raft_consensus.cc:385] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.283926 22290 raft_consensus.cc:740] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.285158 21960 leader_election.cc:304] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:28.287456 22310 raft_consensus.cc:493] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.287041 22290 consensus_queue.cc:260] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.288010 22310 raft_consensus.cc:515] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.288414 22313 raft_consensus.cc:3060] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.296461 22293 tablet_bootstrap.cc:654] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.306058 22289 tablet_bootstrap.cc:654] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.311616 22310 leader_election.cc:290] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.313746 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.314563 22068 raft_consensus.cc:2468] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 0.
I20260709 12:15:28.316284 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.317782 22290 ts_tablet_manager.cc:1434] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.052s	user 0.014s	sys 0.012s
W20260709 12:15:28.318604 22106 leader_election.cc:343] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.319026 22290 tablet_bootstrap.cc:492] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: Bootstrap starting.
I20260709 12:15:28.323925 22109 leader_election.cc:304] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.327135 22310 raft_consensus.cc:2804] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.327636 22310 raft_consensus.cc:493] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.328454 22310 raft_consensus.cc:3060] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.313457 22278 raft_consensus.cc:515] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.353209 22270 raft_consensus.cc:493] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.359467 22270 raft_consensus.cc:515] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.361913 22270 leader_election.cc:290] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.362885 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5583597bdfae45d7bcadd9d354e35f3e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.364768 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5583597bdfae45d7bcadd9d354e35f3e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
W20260709 12:15:28.365154 21960 leader_election.cc:343] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:28.366331 21960 leader_election.cc:343] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.366897 21960 leader_election.cc:304] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:28.368261 22314 raft_consensus.cc:2749] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:28.368687 22278 leader_election.cc:290] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.370613 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267ef64c6085422e98bc0add84f3442f" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:28.371569 22143 raft_consensus.cc:3060] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.373016 22310 raft_consensus.cc:515] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.370755 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267ef64c6085422e98bc0add84f3442f" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:28.376473 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
W20260709 12:15:28.377094 22032 leader_election.cc:343] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.378265 22290 tablet_bootstrap.cc:654] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:28.378414 22106 leader_election.cc:343] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:28.385839 22310 leader_election.cc:290] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:28.390559 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:28.391320 22313 raft_consensus.cc:2749] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:28.393487 21889 catalog_manager.cc:5697] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:28.391397 22068 raft_consensus.cc:3060] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.401445 22290 tablet_bootstrap.cc:492] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: No bootstrap required, opened a new log
I20260709 12:15:28.400696 21891 catalog_manager.cc:5697] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:28.406448 22290 ts_tablet_manager.cc:1403] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.004s
I20260709 12:15:28.410104 22290 raft_consensus.cc:359] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.410892 22290 raft_consensus.cc:385] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.398643 22289 tablet_bootstrap.cc:492] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:28.411324 22290 raft_consensus.cc:740] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.411839 22289 ts_tablet_manager.cc:1403] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.137s	user 0.014s	sys 0.005s
I20260709 12:15:28.412293 22290 consensus_queue.cc:260] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.414683 22290 ts_tablet_manager.cc:1434] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
W20260709 12:15:28.420603 21940 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:28.405005 22143 raft_consensus.cc:2468] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:28.425819 22035 leader_election.cc:304] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.426692 22278 raft_consensus.cc:2804] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.427345 22278 raft_consensus.cc:697] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:28.428340 22278 consensus_queue.cc:237] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.431808 22068 raft_consensus.cc:2468] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 1.
I20260709 12:15:28.422672 22289 raft_consensus.cc:359] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.433358 22289 raft_consensus.cc:385] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.433808 22289 raft_consensus.cc:740] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.434772 22293 tablet_bootstrap.cc:492] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:28.436156 22109 leader_election.cc:304] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.437470 22293 ts_tablet_manager.cc:1403] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.173s	user 0.014s	sys 0.009s
I20260709 12:15:28.438596 22310 raft_consensus.cc:2804] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.439172 22310 raft_consensus.cc:697] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:28.438158 21890 catalog_manager.cc:5697] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:28.440093 22310 consensus_queue.cc:237] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.441354 22293 raft_consensus.cc:359] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.442071 22293 raft_consensus.cc:385] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.442353 22293 raft_consensus.cc:740] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.434803 22289 consensus_queue.cc:260] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.443061 22293 consensus_queue.cc:260] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.446352 22289 ts_tablet_manager.cc:1434] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:15:28.446873 22293 ts_tablet_manager.cc:1434] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:15:28.447687 22289 tablet_bootstrap.cc:492] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:28.448180 22293 tablet_bootstrap.cc:492] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: Bootstrap starting.
I20260709 12:15:28.456316 22289 tablet_bootstrap.cc:654] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.456316 22293 tablet_bootstrap.cc:654] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.464063 21891 catalog_manager.cc:5697] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:28.460412 21888 catalog_manager.cc:5697] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 1 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:28.490937 22289 tablet_bootstrap.cc:492] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:28.491268 22293 tablet_bootstrap.cc:492] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: No bootstrap required, opened a new log
I20260709 12:15:28.491803 22289 ts_tablet_manager.cc:1403] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.044s	user 0.024s	sys 0.014s
I20260709 12:15:28.491923 22293 ts_tablet_manager.cc:1403] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.009s
I20260709 12:15:28.495618 22293 raft_consensus.cc:359] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.496248 22293 raft_consensus.cc:385] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.496587 22293 raft_consensus.cc:740] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.495971 22289 raft_consensus.cc:359] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.497841 22289 raft_consensus.cc:385] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.497587 22293 consensus_queue.cc:260] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.498144 22289 raft_consensus.cc:740] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.499045 22289 consensus_queue.cc:260] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.499794 22293 ts_tablet_manager.cc:1434] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:15:28.501731 22289 ts_tablet_manager.cc:1434] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:15:28.503058 22289 tablet_bootstrap.cc:492] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:28.509438 22289 tablet_bootstrap.cc:654] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.515039 22289 tablet_bootstrap.cc:492] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:28.515566 22289 ts_tablet_manager.cc:1403] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:15:28.518138 22289 raft_consensus.cc:359] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.518687 22289 raft_consensus.cc:385] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.518947 22289 raft_consensus.cc:740] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.519600 22289 consensus_queue.cc:260] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.521735 22289 ts_tablet_manager.cc:1434] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:15:28.523051 22289 tablet_bootstrap.cc:492] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: Bootstrap starting.
I20260709 12:15:28.528503 22289 tablet_bootstrap.cc:654] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:28.533550 22289 tablet_bootstrap.cc:492] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: No bootstrap required, opened a new log
I20260709 12:15:28.534091 22289 ts_tablet_manager.cc:1403] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: Time spent bootstrapping tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:15:28.536619 22289 raft_consensus.cc:359] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.537127 22289 raft_consensus.cc:385] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:28.537297 22289 raft_consensus.cc:740] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Initialized, Role: FOLLOWER
I20260709 12:15:28.537782 22289 consensus_queue.cc:260] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.539443 22289 ts_tablet_manager.cc:1434] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:15:28.542444 22313 raft_consensus.cc:493] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.543051 22313 raft_consensus.cc:515] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.545282 22313 leader_election.cc:290] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.546156 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.547142 22068 raft_consensus.cc:2393] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a89f53d77e4029be05d7a9d2172586 in current term 1: Already voted for candidate 8600e33421784ecb83a556ef90148dd3 in this term.
I20260709 12:15:28.548187 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.549697 21960 leader_election.cc:304] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:28.550884 22313 raft_consensus.cc:3060] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.558216 22313 raft_consensus.cc:2749] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:28.580422 22313 raft_consensus.cc:493] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.580987 22313 raft_consensus.cc:515] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.583292 22313 leader_election.cc:290] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.584251 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.584415 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.585161 22068 raft_consensus.cc:2468] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 0.
I20260709 12:15:28.585233 22143 raft_consensus.cc:2468] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 0.
I20260709 12:15:28.586369 21960 leader_election.cc:304] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.587122 22313 raft_consensus.cc:2804] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.587481 22313 raft_consensus.cc:493] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.587766 22313 raft_consensus.cc:3060] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.594221 22313 raft_consensus.cc:515] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.596110 22313 leader_election.cc:290] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.597123 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:28.597438 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:28.598266 22068 raft_consensus.cc:3060] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.598359 22143 raft_consensus.cc:3060] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.605028 22068 raft_consensus.cc:2468] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 1.
I20260709 12:15:28.606249 21960 leader_election.cc:304] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.607064 22313 raft_consensus.cc:2804] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.606916 22143 raft_consensus.cc:2468] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 1.
I20260709 12:15:28.607733 22313 raft_consensus.cc:697] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:28.609053 22313 consensus_queue.cc:237] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.618532 21888 catalog_manager.cc:5697] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 1 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:28.636349 22278 raft_consensus.cc:493] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.637102 22278 raft_consensus.cc:515] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.639473 22278 leader_election.cc:290] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.640101 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5583597bdfae45d7bcadd9d354e35f3e" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586" is_pre_election: true
I20260709 12:15:28.640692 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5583597bdfae45d7bcadd9d354e35f3e" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.640993 21992 raft_consensus.cc:2468] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:28.641467 22143 raft_consensus.cc:2468] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 0.
I20260709 12:15:28.642680 22032 leader_election.cc:304] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.644330 22278 raft_consensus.cc:2804] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:28.644785 22278 raft_consensus.cc:493] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.645103 22278 raft_consensus.cc:3060] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.646256 22310 consensus_queue.cc:1048] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:28.654008 22278 raft_consensus.cc:515] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.658109 22278 leader_election.cc:290] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.658414 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5583597bdfae45d7bcadd9d354e35f3e" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:28.658784 21993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5583597bdfae45d7bcadd9d354e35f3e" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:28.659286 22143 raft_consensus.cc:3060] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.659566 21993 raft_consensus.cc:3060] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.660984 22310 consensus_queue.cc:1048] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:28.677000 22313 raft_consensus.cc:493] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.677572 22313 raft_consensus.cc:515] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.680338 22278 consensus_queue.cc:1048] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.684160 21993 raft_consensus.cc:2468] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:28.685824 22032 leader_election.cc:304] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:28.686291 22143 raft_consensus.cc:2468] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 1.
I20260709 12:15:28.687194 22278 raft_consensus.cc:2804] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:28.688166 22278 raft_consensus.cc:697] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:28.688776 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.689723 22068 raft_consensus.cc:2393] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a89f53d77e4029be05d7a9d2172586 in current term 1: Already voted for candidate 8600e33421784ecb83a556ef90148dd3 in this term.
I20260709 12:15:28.689626 22313 leader_election.cc:290] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.689533 22278 consensus_queue.cc:237] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.690896 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.692662 21960 leader_election.cc:304] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:28.694221 22313 raft_consensus.cc:3060] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.705366 22329 consensus_queue.cc:1048] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.705885 22313 raft_consensus.cc:2749] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:28.713339 21888 catalog_manager.cc:5697] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 1 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:28.718246 22310 consensus_queue.cc:1048] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.734403 22315 consensus_queue.cc:1048] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.744927 22314 consensus_queue.cc:1048] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.746399 22270 consensus_queue.cc:1048] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:28.775338 22313 consensus_queue.cc:1048] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.794310 22270 raft_consensus.cc:493] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.795094 22270 raft_consensus.cc:515] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:28.797317 22314 consensus_queue.cc:1048] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.802263 22069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267ef64c6085422e98bc0add84f3442f" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8" is_pre_election: true
I20260709 12:15:28.804900 22270 leader_election.cc:290] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:28.806321 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "267ef64c6085422e98bc0add84f3442f" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8600e33421784ecb83a556ef90148dd3" is_pre_election: true
I20260709 12:15:28.807279 22143 raft_consensus.cc:2393] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e2a89f53d77e4029be05d7a9d2172586 in current term 1: Already voted for candidate 738c8826f8f5430d937cdbb7805bdfb8 in this term.
I20260709 12:15:28.808636 21960 leader_election.cc:304] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e2a89f53d77e4029be05d7a9d2172586; no voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:28.809399 22270 raft_consensus.cc:3060] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:28.818940 22270 raft_consensus.cc:2749] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:28.856038 22315 consensus_queue.cc:1048] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:28.864112 22315 consensus_queue.cc:1048] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:28.911741 22141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:28.912465 22141 raft_consensus.cc:606] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.918241 22068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36940
I20260709 12:15:28.918764 22068 raft_consensus.cc:606] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.926636 22141 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:28.927278 22141 raft_consensus.cc:606] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS e2a89f53d77e4029be05d7a9d2172586
I20260709 12:15:28.934279 22329 consensus_queue.cc:1048] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:28.935629 22143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8de6bdba9564589947ac9874077e391"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
mode: GRACEFUL
new_leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:51398
I20260709 12:15:28.936367 22143 raft_consensus.cc:606] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Received request to transfer leadership to TS 738c8826f8f5430d937cdbb7805bdfb8
I20260709 12:15:28.943208 21993 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
mode: GRACEFUL
new_leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36418
I20260709 12:15:28.943941 21993 raft_consensus.cc:606] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Received request to transfer leadership to TS 738c8826f8f5430d937cdbb7805bdfb8
I20260709 12:15:28.946201 22278 consensus_queue.cc:1048] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:28.960947 21992 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
mode: GRACEFUL
new_leader_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:36418
I20260709 12:15:28.961630 21992 raft_consensus.cc:606] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Received request to transfer leadership to TS 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:29.018656 22314 consensus_queue.cc:1048] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:29.026731 22270 raft_consensus.cc:993] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: : Instructing follower 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:29.027657 22313 raft_consensus.cc:1081] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Signalling peer 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:29.029668 22143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:51396
I20260709 12:15:29.030388 22143 raft_consensus.cc:493] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.030786 22143 raft_consensus.cc:3060] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.034207 22313 consensus_queue.cc:1048] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:29.042215 22143 raft_consensus.cc:515] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.045303 21992 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:29.046114 21992 raft_consensus.cc:3055] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.046487 21992 raft_consensus.cc:740] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:29.046548 22143 leader_election.cc:290] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:29.046744 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea14aee8ecd84228ab92126aa2243131" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:29.047432 21992 consensus_queue.cc:260] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.047490 22068 raft_consensus.cc:3060] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.049080 21992 raft_consensus.cc:3060] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.057102 21992 raft_consensus.cc:2468] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 2.
I20260709 12:15:29.057471 22068 raft_consensus.cc:2468] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 2.
I20260709 12:15:29.058367 22106 leader_election.cc:304] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:29.059438 22315 raft_consensus.cc:2804] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.059935 22315 raft_consensus.cc:697] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 2 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:29.060797 22315 consensus_queue.cc:237] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.069284 21892 catalog_manager.cc:5697] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 1 to 2, leader changed from e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65) to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 2 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:29.103609 22291 raft_consensus.cc:993] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:29.104180 22315 raft_consensus.cc:1081] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:29.105684 21992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36408
I20260709 12:15:29.106226 21992 raft_consensus.cc:493] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.106542 21992 raft_consensus.cc:3060] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.112555 22278 consensus_queue.cc:1048] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:29.113619 21992 raft_consensus.cc:515] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.116091 21992 leader_election.cc:290] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:29.117506 22067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:29.118245 22067 raft_consensus.cc:3060] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.119411 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5da0d2a09f547a6a9457178dcd02a85" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:29.120189 22141 raft_consensus.cc:3055] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.120515 22141 raft_consensus.cc:740] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:29.121483 22141 consensus_queue.cc:260] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.122725 22141 raft_consensus.cc:3060] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.128181 22067 raft_consensus.cc:2468] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:29.129814 21960 leader_election.cc:304] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:29.130786 22313 raft_consensus.cc:2804] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.131314 22313 raft_consensus.cc:697] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:29.131559 22329 raft_consensus.cc:993] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:29.132133 22329 raft_consensus.cc:1081] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:29.132236 22313 consensus_queue.cc:237] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.135673 22141 raft_consensus.cc:2468] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:29.139300 22278 consensus_queue.cc:1048] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:29.146777 22315 raft_consensus.cc:993] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: : Instructing follower e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:29.147665 22312 raft_consensus.cc:1081] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer e2a89f53d77e4029be05d7a9d2172586 to start an election
I20260709 12:15:29.147758 21891 catalog_manager.cc:5697] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.149957 21994 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36408
I20260709 12:15:29.150534 21994 raft_consensus.cc:493] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.150893 21994 raft_consensus.cc:3060] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.160396 21992 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
 from {username='slave'} at 127.0.0.1:36396
I20260709 12:15:29.161227 21992 raft_consensus.cc:493] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.162384 21992 raft_consensus.cc:3060] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.162496 21994 raft_consensus.cc:515] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.165968 22067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:29.166764 22067 raft_consensus.cc:3060] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.170475 21994 leader_election.cc:290] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:29.171801 22143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12363c88ac98469589a4893c96df2d5e" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:29.172536 22143 raft_consensus.cc:3055] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.172824 22143 raft_consensus.cc:740] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:29.173069 21992 raft_consensus.cc:515] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.173724 22143 consensus_queue.cc:260] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.175199 22143 raft_consensus.cc:3060] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.175642 21992 leader_election.cc:290] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Requested vote from peers 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:29.176070 22067 raft_consensus.cc:2468] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:29.177304 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:29.177578 21960 leader_election.cc:304] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:29.177839 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d74c6d0fa6e943c58de2cc7643a98d93" candidate_uuid: "e2a89f53d77e4029be05d7a9d2172586" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:29.178579 22141 raft_consensus.cc:3060] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.178649 22313 raft_consensus.cc:2804] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.179283 22313 raft_consensus.cc:697] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:29.180197 22313 consensus_queue.cc:237] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.177977 22068 raft_consensus.cc:3055] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.182922 22143 raft_consensus.cc:2468] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:29.183151 22068 raft_consensus.cc:740] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:29.184137 22068 consensus_queue.cc:260] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.185489 22068 raft_consensus.cc:3060] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.190404 21891 catalog_manager.cc:5697] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.192019 22068 raft_consensus.cc:2468] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:29.192219 22141 raft_consensus.cc:2468] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2a89f53d77e4029be05d7a9d2172586 in term 2.
I20260709 12:15:29.193329 21960 leader_election.cc:304] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:29.194131 22313 raft_consensus.cc:2804] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.194623 22313 raft_consensus.cc:697] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Leader. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:29.195451 22313 consensus_queue.cc:237] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.205981 21891 catalog_manager.cc:5697] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 reported cstate change: term changed from 1 to 2, leader changed from 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66) to e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65). New cstate: current_term: 2 leader_uuid: "e2a89f53d77e4029be05d7a9d2172586" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.279196 22270 raft_consensus.cc:993] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: : Instructing follower 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:29.279772 22313 raft_consensus.cc:1081] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Signalling peer 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:29.281467 22068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36912
I20260709 12:15:29.282013 22068 raft_consensus.cc:493] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.282344 22068 raft_consensus.cc:3060] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.288638 22068 raft_consensus.cc:515] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.290611 22068 leader_election.cc:290] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:29.291402 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:29.291967 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a53af6bb389648d6b3f53d229d48e9b7" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:29.292189 21994 raft_consensus.cc:3055] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.292670 21994 raft_consensus.cc:740] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:29.292711 22141 raft_consensus.cc:3060] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.293534 21994 consensus_queue.cc:260] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.294634 21994 raft_consensus.cc:3060] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.299614 22141 raft_consensus.cc:2468] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:29.300531 21994 raft_consensus.cc:2468] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:29.301059 22035 leader_election.cc:304] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, 8600e33421784ecb83a556ef90148dd3; no voters: 
I20260709 12:15:29.302083 22311 raft_consensus.cc:2804] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.302577 22311 raft_consensus.cc:697] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:29.303431 22311 consensus_queue.cc:237] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.311383 21891 catalog_manager.cc:5697] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 1 to 2, leader changed from e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65) to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.440986 22277 raft_consensus.cc:993] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: : Instructing follower 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:29.441597 22315 raft_consensus.cc:1081] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Signalling peer 738c8826f8f5430d937cdbb7805bdfb8 to start an election
I20260709 12:15:29.443326 22068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8de6bdba9564589947ac9874077e391"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
 from {username='slave'} at 127.0.0.1:36928
I20260709 12:15:29.443890 22068 raft_consensus.cc:493] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:29.444283 22068 raft_consensus.cc:3060] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.451141 22068 raft_consensus.cc:515] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.452679 22068 leader_election.cc:290] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 8600e33421784ecb83a556ef90148dd3 (127.9.11.67:33391)
I20260709 12:15:29.454030 22141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8600e33421784ecb83a556ef90148dd3"
I20260709 12:15:29.454490 21994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8de6bdba9564589947ac9874077e391" candidate_uuid: "738c8826f8f5430d937cdbb7805bdfb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:29.454934 22141 raft_consensus.cc:3055] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:15:29.455291 21994 raft_consensus.cc:3060] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.455474 22141 raft_consensus.cc:740] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:29.457597 22141 consensus_queue.cc:260] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.459560 22141 raft_consensus.cc:3060] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:29.463722 21994 raft_consensus.cc:2468] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:29.465080 22032 leader_election.cc:304] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 738c8826f8f5430d937cdbb7805bdfb8, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:29.465947 22329 raft_consensus.cc:2804] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:29.466400 22329 raft_consensus.cc:697] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Becoming Leader. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:29.467315 22141 raft_consensus.cc:2468] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 738c8826f8f5430d937cdbb7805bdfb8 in term 2.
I20260709 12:15:29.467283 22329 consensus_queue.cc:237] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:29.475484 21891 catalog_manager.cc:5697] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 reported cstate change: term changed from 1 to 2, leader changed from 8600e33421784ecb83a556ef90148dd3 (127.9.11.67) to 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66). New cstate: current_term: 2 leader_uuid: "738c8826f8f5430d937cdbb7805bdfb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.518837 22068 raft_consensus.cc:1275] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.520531 22315 consensus_queue.cc:1048] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.529134 21994 raft_consensus.cc:1275] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.530927 22312 consensus_queue.cc:1048] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.603489 22141 raft_consensus.cc:1275] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.604959 22313 consensus_queue.cc:1048] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.610569 22068 raft_consensus.cc:1275] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.613224 22313 consensus_queue.cc:1048] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.681461 22141 raft_consensus.cc:1275] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.682945 22313 consensus_queue.cc:1048] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.691033 22068 raft_consensus.cc:1275] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.692698 22270 consensus_queue.cc:1048] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.715481 22068 raft_consensus.cc:1275] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.717108 22364 consensus_queue.cc:1048] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.730911 22141 raft_consensus.cc:1275] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer e2a89f53d77e4029be05d7a9d2172586: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.732844 22313 consensus_queue.cc:1048] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:15:29.759222 22141 raft_consensus.cc:1275] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.760617 22329 consensus_queue.cc:1048] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.770781 21994 raft_consensus.cc:1275] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.772801 22278 consensus_queue.cc:1048] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.879492 22141 raft_consensus.cc:1275] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.880966 22278 consensus_queue.cc:1048] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:15:29.891413 21994 raft_consensus.cc:1275] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Refusing update from remote peer 738c8826f8f5430d937cdbb7805bdfb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:15:29.893447 22278 consensus_queue.cc:1048] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:15:31.248258 21923 debug-util.cc:398] Leaking SignalData structure 0x7b0800394760 after lost signal to thread 9264
W20260709 12:15:31.249805 21923 debug-util.cc:398] Leaking SignalData structure 0x7b08001bf280 after lost signal to thread 21926
W20260709 12:15:31.251367 21923 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca860 after lost signal to thread 22018
W20260709 12:15:31.252710 21923 debug-util.cc:398] Leaking SignalData structure 0x7b0800279400 after lost signal to thread 22093
W20260709 12:15:31.254000 21923 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2dc0 after lost signal to thread 22167
I20260709 12:15:32.011819 21891 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38986:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:15:32.044392 21891 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38986:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:15:32.185333  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:32.241966 22395 log.cc:927] Time spent T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.001s
W20260709 12:15:32.300598 21960 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44195, user_credentials={real_user=slave}} blocked reactor thread for 56096.4us
W20260709 12:15:32.297307 22400 log.cc:927] Time spent T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:15:32.329563 22416 tablet.cc:1724] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.386894 22427 tablet.cc:1724] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:15:32.428066 21960 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.11.66:44195, user_credentials={real_user=slave}} blocked reactor thread for 99695.5us
I20260709 12:15:32.392452 22421 tablet.cc:1724] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.432523 22422 tablet.cc:1724] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.469483 22432 tablet.cc:1724] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.491084 22435 tablet.cc:1724] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.489886 22442 tablet.cc:1724] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.510525 22440 tablet.cc:1724] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.520423 22068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f84307224ba4c489d4120bc91039178"
dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
mode: GRACEFUL
new_leader_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:59524
I20260709 12:15:32.521272 22068 raft_consensus.cc:606] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Received request to transfer leadership to TS 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:32.542968 22438 tablet.cc:1724] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:15:32.553826 22433 log.cc:927] Time spent T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:15:32.560266 22441 tablet.cc:1724] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.564136 22440 tablet.cc:1724] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.568259 22421 tablet.cc:1724] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.575938 22444 tablet.cc:1724] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.607360 22432 tablet.cc:1724] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.610524 22443 tablet.cc:1724] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.591802 22416 tablet.cc:1724] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.632088 22444 tablet.cc:1724] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.636211 22435 tablet.cc:1724] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.652076 22440 tablet.cc:1724] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.663113 22422 tablet.cc:1724] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.665053 22427 tablet.cc:1724] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.665633 22442 tablet.cc:1724] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.668056 22441 tablet.cc:1724] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.684303 22443 tablet.cc:1724] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.696777 21991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029"
dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
mode: GRACEFUL
new_leader_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:42658
I20260709 12:15:32.697708 21991 raft_consensus.cc:606] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Received request to transfer leadership to TS 8600e33421784ecb83a556ef90148dd3
I20260709 12:15:32.699968  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:15:32.700789  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:32.686899 22416 tablet.cc:1724] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.693073 22438 tablet.cc:1724] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.712795 22421 tablet.cc:1724] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:15:32.765667 21892 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac: soft_deleted_table [id=8eb1808ea4a74aaa92e717906c0d0fa3] alter complete (version 1)
I20260709 12:15:32.918108 22439 raft_consensus.cc:993] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: : Instructing follower 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:32.918874 22437 raft_consensus.cc:1081] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Signalling peer 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:32.921013 22137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f84307224ba4c489d4120bc91039178"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:51388
I20260709 12:15:32.921648 22137 raft_consensus.cc:493] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.922063 22137 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.930392 22137 raft_consensus.cc:515] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:32.932687 22137 leader_election.cc:290] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 3 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:32.933578 21991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:32.933894 22068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f84307224ba4c489d4120bc91039178" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:32.934480 21991 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.934579 22068 raft_consensus.cc:3055] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:32.935031 22068 raft_consensus.cc:740] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 738c8826f8f5430d937cdbb7805bdfb8, State: Running, Role: LEADER
I20260709 12:15:32.935938 22068 consensus_queue.cc:260] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:32.937536 22068 raft_consensus.cc:3060] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.944576 21991 raft_consensus.cc:2468] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 3.
I20260709 12:15:32.946396 22106 leader_election.cc:304] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:32.946651 22068 raft_consensus.cc:2468] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 3.
I20260709 12:15:32.946434 22364 raft_consensus.cc:993] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: : Instructing follower 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:32.947540 22364 raft_consensus.cc:1081] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Signalling peer 8600e33421784ecb83a556ef90148dd3 to start an election
I20260709 12:15:32.947710 22425 raft_consensus.cc:2804] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:32.948366 22425 raft_consensus.cc:697] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:32.949592 22137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029"
dest_uuid: "8600e33421784ecb83a556ef90148dd3"
 from {username='slave'} at 127.0.0.1:51396
I20260709 12:15:32.949337 22425 consensus_queue.cc:237] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:32.950263 22137 raft_consensus.cc:493] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:32.960553 22137 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.968529 22137 raft_consensus.cc:515] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:32.970650 22137 leader_election.cc:290] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 3 election: Requested vote from peers e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861), 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195)
I20260709 12:15:32.971635 21991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e2a89f53d77e4029be05d7a9d2172586"
I20260709 12:15:32.972344 21991 raft_consensus.cc:3055] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:15:32.972656 21991 raft_consensus.cc:740] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e2a89f53d77e4029be05d7a9d2172586, State: Running, Role: LEADER
I20260709 12:15:32.973631 21991 consensus_queue.cc:260] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:32.974725 21991 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.980686 22065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a19e3221c5de4c38802bdebb3994c029" candidate_uuid: "8600e33421784ecb83a556ef90148dd3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "738c8826f8f5430d937cdbb7805bdfb8"
I20260709 12:15:32.981544 22065 raft_consensus.cc:3060] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:15:32.984177 21991 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 3.
I20260709 12:15:32.985749 22106 leader_election.cc:304] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8600e33421784ecb83a556ef90148dd3, e2a89f53d77e4029be05d7a9d2172586; no voters: 
I20260709 12:15:32.986863 22425 raft_consensus.cc:2804] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:15:32.987428 22425 raft_consensus.cc:697] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 3 LEADER]: Becoming Leader. State: Replica: 8600e33421784ecb83a556ef90148dd3, State: Running, Role: LEADER
I20260709 12:15:32.988309 22425 consensus_queue.cc:237] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } }
I20260709 12:15:32.994334 22065 raft_consensus.cc:2468] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8600e33421784ecb83a556ef90148dd3 in term 3.
I20260709 12:15:33.006254 21890 catalog_manager.cc:5697] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 2 to 3, leader changed from 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66) to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.008684 21890 catalog_manager.cc:5697] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 reported cstate change: term changed from 2 to 3, leader changed from e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65) to 8600e33421784ecb83a556ef90148dd3 (127.9.11.67). New cstate: current_term: 3 leader_uuid: "8600e33421784ecb83a556ef90148dd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8600e33421784ecb83a556ef90148dd3" member_type: VOTER last_known_addr { host: "127.9.11.67" port: 33391 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.369902 21991 raft_consensus.cc:1275] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.371490 22425 consensus_queue.cc:1048] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.389922 22065 raft_consensus.cc:1275] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.391623 22425 consensus_queue.cc:1048] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.475435 22068 raft_consensus.cc:1275] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.476868 22425 consensus_queue.cc:1048] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "738c8826f8f5430d937cdbb7805bdfb8" member_type: VOTER last_known_addr { host: "127.9.11.66" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:15:33.490518 21991 raft_consensus.cc:1275] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Refusing update from remote peer 8600e33421784ecb83a556ef90148dd3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:15:33.492663 22458 consensus_queue.cc:1048] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a89f53d77e4029be05d7a9d2172586" member_type: VOTER last_known_addr { host: "127.9.11.65" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:15:33.702193  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:33.708425  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:33.709008  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:34.710405  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:34.716656  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:34.717278  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:35.718675  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:35.726668  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:35.727347  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:36.728657  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:36.734268  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:36.734874  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:37.736099  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:37.741650  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:37.742208  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:38.743635  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:38.749408  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:38.750002  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:39.751176  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:39.758023  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:39.758567  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:40.760046  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:40.766301  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:40.766891  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:41.768038  9261 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:41.775671  9261 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:15:41.776250  9261 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:42.845383  9261 tablet_server.cc:179] TabletServer@127.9.11.65:0 shutting down...
W20260709 12:15:42.864419 22106 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20260709 12:15:42.877020 22106 consensus_peers.cc:597] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:42.881836 22032 consensus_peers.cc:597] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:42.897344 22032 consensus_peers.cc:597] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:42.912106  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:42.913321  9261 tablet_replica.cc:333] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.915103  9261 raft_consensus.cc:2243] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.916508  9261 raft_consensus.cc:2272] T e8de6bdba9564589947ac9874077e391 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.920565  9261 tablet_replica.cc:333] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.921617  9261 raft_consensus.cc:2243] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.922533  9261 raft_consensus.cc:2272] T 5583597bdfae45d7bcadd9d354e35f3e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.926390  9261 tablet_replica.cc:333] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.927451  9261 raft_consensus.cc:2243] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.928141  9261 raft_consensus.cc:2272] T ea14aee8ecd84228ab92126aa2243131 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.932081  9261 tablet_replica.cc:333] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.933127  9261 raft_consensus.cc:2243] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:15:42.933219 22106 consensus_peers.cc:597] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:42.934496  9261 raft_consensus.cc:2272] T c98fca4dbfcf412499ce812e14ca0a4e P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:42.940964 22106 consensus_peers.cc:597] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:42.938715  9261 tablet_replica.cc:333] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.942699  9261 raft_consensus.cc:2243] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.943641  9261 raft_consensus.cc:2272] T 8f84307224ba4c489d4120bc91039178 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.947484  9261 tablet_replica.cc:333] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.948221  9261 raft_consensus.cc:2243] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.948909  9261 raft_consensus.cc:2272] T a53af6bb389648d6b3f53d229d48e9b7 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.952417  9261 tablet_replica.cc:333] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.953411  9261 raft_consensus.cc:2243] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.954226  9261 raft_consensus.cc:2272] T 8c965fb2a6f24b2eace0008b8f83bbbb P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.957353  9261 tablet_replica.cc:333] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.958019  9261 raft_consensus.cc:2243] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.959156  9261 raft_consensus.cc:2272] T 12363c88ac98469589a4893c96df2d5e P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.961964  9261 tablet_replica.cc:333] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.962669  9261 raft_consensus.cc:2243] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.963281  9261 raft_consensus.cc:2272] T 335b3b26e87f404fa95b6d2a3c85f605 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.966202  9261 tablet_replica.cc:333] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.966825  9261 raft_consensus.cc:2243] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.967990  9261 raft_consensus.cc:2272] T 4cbb5d889cc3467e9a166982b4eacfe0 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.970906  9261 tablet_replica.cc:333] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.971689  9261 raft_consensus.cc:2243] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.972193  9261 raft_consensus.cc:2272] T 2f55e2d5913649e6a00e298a453cc96a P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.975009  9261 tablet_replica.cc:333] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.975565  9261 raft_consensus.cc:2243] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.976651  9261 raft_consensus.cc:2272] T 1fcbeb90ec3641b88309d64a33a7fe90 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.979490  9261 tablet_replica.cc:333] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.980150  9261 raft_consensus.cc:2243] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.981220  9261 raft_consensus.cc:2272] T d74c6d0fa6e943c58de2cc7643a98d93 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.984354  9261 tablet_replica.cc:333] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.985203  9261 raft_consensus.cc:2243] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:42.986677  9261 raft_consensus.cc:2272] T 2339e13a2b06478b8e08f568b25762a1 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.990626  9261 tablet_replica.cc:333] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.991544  9261 raft_consensus.cc:2243] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.992100  9261 raft_consensus.cc:2272] T 267ef64c6085422e98bc0add84f3442f P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.994760  9261 tablet_replica.cc:333] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.995390  9261 raft_consensus.cc:2243] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:42.995889  9261 raft_consensus.cc:2272] T 673096df60ed44bc8b2a9856e6e1b884 P e2a89f53d77e4029be05d7a9d2172586 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:42.998930  9261 tablet_replica.cc:333] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:42.999516  9261 raft_consensus.cc:2243] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.000571  9261 raft_consensus.cc:2272] T f5da0d2a09f547a6a9457178dcd02a85 P e2a89f53d77e4029be05d7a9d2172586 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.003257  9261 tablet_replica.cc:333] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586: stopping tablet replica
I20260709 12:15:43.003821  9261 raft_consensus.cc:2243] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.004344  9261 raft_consensus.cc:2272] T a19e3221c5de4c38802bdebb3994c029 P e2a89f53d77e4029be05d7a9d2172586 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:43.051548 22032 consensus_peers.cc:597] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:46861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:43.058794 22032 consensus_peers.cc:597] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:46861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:43.061482  9261 tablet_server.cc:196] TabletServer@127.9.11.65:0 shutdown complete.
I20260709 12:15:43.124583  9261 tablet_server.cc:179] TabletServer@127.9.11.66:0 shutting down...
I20260709 12:15:43.168097  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:43.169245  9261 tablet_replica.cc:333] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
W20260709 12:15:43.169214 22032 consensus_peers.cc:597] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:46861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:43.170650  9261 raft_consensus.cc:2243] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.171824  9261 raft_consensus.cc:2272] T 5583597bdfae45d7bcadd9d354e35f3e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.174787  9261 tablet_replica.cc:333] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.175633  9261 raft_consensus.cc:2243] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.176703  9261 raft_consensus.cc:2272] T e8de6bdba9564589947ac9874077e391 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.179471  9261 tablet_replica.cc:333] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.180104  9261 raft_consensus.cc:2243] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.180711  9261 raft_consensus.cc:2272] T ea14aee8ecd84228ab92126aa2243131 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.183324  9261 tablet_replica.cc:333] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.183991  9261 raft_consensus.cc:2243] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.184590  9261 raft_consensus.cc:2272] T c98fca4dbfcf412499ce812e14ca0a4e P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.187366  9261 tablet_replica.cc:333] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.187968  9261 raft_consensus.cc:2243] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.188529  9261 raft_consensus.cc:2272] T 2339e13a2b06478b8e08f568b25762a1 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.191188  9261 tablet_replica.cc:333] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.191964  9261 raft_consensus.cc:2243] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.192699  9261 raft_consensus.cc:2272] T d74c6d0fa6e943c58de2cc7643a98d93 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.196204  9261 tablet_replica.cc:333] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.196981  9261 raft_consensus.cc:2243] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.197757  9261 raft_consensus.cc:2272] T f5da0d2a09f547a6a9457178dcd02a85 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.201025  9261 tablet_replica.cc:333] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.201753  9261 raft_consensus.cc:2243] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.202327  9261 raft_consensus.cc:2272] T a19e3221c5de4c38802bdebb3994c029 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.204929  9261 tablet_replica.cc:333] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.205618  9261 raft_consensus.cc:2243] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.206633  9261 raft_consensus.cc:2272] T a53af6bb389648d6b3f53d229d48e9b7 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.209298  9261 tablet_replica.cc:333] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.209887  9261 raft_consensus.cc:2243] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.210391  9261 raft_consensus.cc:2272] T 8f84307224ba4c489d4120bc91039178 P 738c8826f8f5430d937cdbb7805bdfb8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.213023  9261 tablet_replica.cc:333] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.213621  9261 raft_consensus.cc:2243] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.214597  9261 raft_consensus.cc:2272] T 8c965fb2a6f24b2eace0008b8f83bbbb P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.217432  9261 tablet_replica.cc:333] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.217979  9261 raft_consensus.cc:2243] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.218502  9261 raft_consensus.cc:2272] T 12363c88ac98469589a4893c96df2d5e P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.221138  9261 tablet_replica.cc:333] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.221730  9261 raft_consensus.cc:2243] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.222205  9261 raft_consensus.cc:2272] T 335b3b26e87f404fa95b6d2a3c85f605 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.224752  9261 tablet_replica.cc:333] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.225301  9261 raft_consensus.cc:2243] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.225767  9261 raft_consensus.cc:2272] T 4cbb5d889cc3467e9a166982b4eacfe0 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.229030  9261 tablet_replica.cc:333] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.229833  9261 raft_consensus.cc:2243] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.231137  9261 raft_consensus.cc:2272] T 2f55e2d5913649e6a00e298a453cc96a P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.233775  9261 tablet_replica.cc:333] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.234368  9261 raft_consensus.cc:2243] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.234833  9261 raft_consensus.cc:2272] T 1fcbeb90ec3641b88309d64a33a7fe90 P 738c8826f8f5430d937cdbb7805bdfb8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.237429  9261 tablet_replica.cc:333] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.237915  9261 raft_consensus.cc:2243] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.238883  9261 raft_consensus.cc:2272] T 267ef64c6085422e98bc0add84f3442f P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.241463  9261 tablet_replica.cc:333] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8: stopping tablet replica
I20260709 12:15:43.241919  9261 raft_consensus.cc:2243] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.242872  9261 raft_consensus.cc:2272] T 673096df60ed44bc8b2a9856e6e1b884 P 738c8826f8f5430d937cdbb7805bdfb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.301760  9261 tablet_server.cc:196] TabletServer@127.9.11.66:0 shutdown complete.
I20260709 12:15:43.357227  9261 tablet_server.cc:179] TabletServer@127.9.11.67:0 shutting down...
W20260709 12:15:43.361235 22106 consensus_peers.cc:597] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 -> Peer e2a89f53d77e4029be05d7a9d2172586 (127.9.11.65:46861): Couldn't send request to peer e2a89f53d77e4029be05d7a9d2172586. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.65:46861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:43.402858  9261 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:43.403928  9261 tablet_replica.cc:333] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.404860  9261 raft_consensus.cc:2243] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.405664  9261 raft_consensus.cc:2272] T 5583597bdfae45d7bcadd9d354e35f3e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.408670  9261 tablet_replica.cc:333] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.409456  9261 raft_consensus.cc:2243] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.410089  9261 raft_consensus.cc:2272] T e8de6bdba9564589947ac9874077e391 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.412876  9261 tablet_replica.cc:333] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.413578  9261 raft_consensus.cc:2243] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.414642  9261 raft_consensus.cc:2272] T ea14aee8ecd84228ab92126aa2243131 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.417922  9261 tablet_replica.cc:333] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.418650  9261 raft_consensus.cc:2243] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.419381  9261 raft_consensus.cc:2272] T c98fca4dbfcf412499ce812e14ca0a4e P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.422142  9261 tablet_replica.cc:333] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.422859  9261 raft_consensus.cc:2243] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.424005  9261 raft_consensus.cc:2272] T 335b3b26e87f404fa95b6d2a3c85f605 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.426892  9261 tablet_replica.cc:333] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.427681  9261 raft_consensus.cc:2243] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.428283  9261 raft_consensus.cc:2272] T 12363c88ac98469589a4893c96df2d5e P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.430927  9261 tablet_replica.cc:333] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.431643  9261 raft_consensus.cc:2243] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.432160  9261 raft_consensus.cc:2272] T a53af6bb389648d6b3f53d229d48e9b7 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.434667  9261 tablet_replica.cc:333] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.435350  9261 raft_consensus.cc:2243] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.436344  9261 raft_consensus.cc:2272] T 8f84307224ba4c489d4120bc91039178 P 8600e33421784ecb83a556ef90148dd3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.438903  9261 tablet_replica.cc:333] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.439615  9261 raft_consensus.cc:2243] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.440212  9261 raft_consensus.cc:2272] T 8c965fb2a6f24b2eace0008b8f83bbbb P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.443692  9261 tablet_replica.cc:333] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.444628  9261 raft_consensus.cc:2243] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:15:43.445044 22109 consensus_peers.cc:597] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 -> Peer 738c8826f8f5430d937cdbb7805bdfb8 (127.9.11.66:44195): Couldn't send request to peer 738c8826f8f5430d937cdbb7805bdfb8. Status: Network error: Client connection negotiation failed: client connection to 127.9.11.66:44195: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:43.445511  9261 raft_consensus.cc:2272] T f5da0d2a09f547a6a9457178dcd02a85 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.449070  9261 tablet_replica.cc:333] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.449827  9261 raft_consensus.cc:2243] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.450894  9261 raft_consensus.cc:2272] T a19e3221c5de4c38802bdebb3994c029 P 8600e33421784ecb83a556ef90148dd3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.453675  9261 tablet_replica.cc:333] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.454366  9261 raft_consensus.cc:2243] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.454958  9261 raft_consensus.cc:2272] T 267ef64c6085422e98bc0add84f3442f P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.457612  9261 tablet_replica.cc:333] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.458263  9261 raft_consensus.cc:2243] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.458787  9261 raft_consensus.cc:2272] T 673096df60ed44bc8b2a9856e6e1b884 P 8600e33421784ecb83a556ef90148dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.461350  9261 tablet_replica.cc:333] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.461977  9261 raft_consensus.cc:2243] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.462522  9261 raft_consensus.cc:2272] T 1fcbeb90ec3641b88309d64a33a7fe90 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.465427  9261 tablet_replica.cc:333] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.466161  9261 raft_consensus.cc:2243] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.466714  9261 raft_consensus.cc:2272] T 2f55e2d5913649e6a00e298a453cc96a P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.469295  9261 tablet_replica.cc:333] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.469776  9261 raft_consensus.cc:2243] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.470243  9261 raft_consensus.cc:2272] T 4cbb5d889cc3467e9a166982b4eacfe0 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.472738  9261 tablet_replica.cc:333] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.473232  9261 raft_consensus.cc:2243] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.473707  9261 raft_consensus.cc:2272] T d74c6d0fa6e943c58de2cc7643a98d93 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.476204  9261 tablet_replica.cc:333] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3: stopping tablet replica
I20260709 12:15:43.476665  9261 raft_consensus.cc:2243] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:43.477134  9261 raft_consensus.cc:2272] T 2339e13a2b06478b8e08f568b25762a1 P 8600e33421784ecb83a556ef90148dd3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.528554  9261 tablet_server.cc:196] TabletServer@127.9.11.67:0 shutdown complete.
I20260709 12:15:43.577880  9261 master.cc:562] Master@127.9.11.126:34477 shutting down...
I20260709 12:15:43.611105  9261 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:43.611888  9261 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:43.612212  9261 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8d4023a6dc134583a76269f52824ceac: stopping tablet replica
I20260709 12:15:43.656945  9261 master.cc:584] Master@127.9.11.126:34477 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (21631 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (42886 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (449515 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:15:43.727252  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2229 messages since previous log ~33 seconds ago
I20260709 12:15:43.727556  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2228 messages since previous log ~33 seconds ago
I20260709 12:15:43.727752  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2225 messages since previous log ~33 seconds ago
I20260709 12:15:43.728004  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:15:43.728212  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:15:43.728595  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 12:15:43.728811  9261 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~20 seconds ago
